CONSTRUCT VALIDITY LESSON PLAN
Step 1: 10 minutes
o Break students into groups and distribute group worksheets (direct students to introduce selves to group members)
o Describe basic task:
- students will be creating “mini” tests of depression (as they define it) and proposing research studies to determine if their tests actually do a good job in measuring depression
o Additional directions
- Take individual notes so that you have the information (you will be handing in the group worksheet) – accommodations for the “recorder”
- Be sure everyone reads their definition out loud
- Your group definition does not have to be “the perfect” definition, given time limit
- You will have about 10 minutes to complete this step
Step 2: 10 minutes
o Instruct groups to finish definitions and begin writing items
o Items should related to their definitions/essential characteristics of a depressed person
o Review Likert scale anchors
o Remind them to spend about 10 minutes to complete step
Step 3: 30 to 40 minutes
o Instruct groups to finish writing items and begin to propose research studies to determine if their tests actually measures depression
o Remind them:
- Unlimited resources – funding, people from a variety of settings (e.g. clinics, university, whatever), ages, diagnoses, etc.
- Think about your definitions of depression, what you know about depression as you think about what results you might expect
o Monitor groups progress; if groups seem to be going well, let them continue
o If groups struggling, off track call groups attention
- Pick some examples to review
Step 4: 15 minutes
o Call for groups attention and provide instructions for final method
o Distribute additional measures
o Think about how you might use these tests to provide evidence of the ability of your test to measure depression well
- Hint: think about the statistical methods we have covered in class to this point
Step 5: 10 minutes
o After all groups have completed the task, if time, discuss some of the definitions and methods
o Have group members describe as worksheets are displayed on visualizer
o Save 5 minutes at end (if possible) for writing exercise:
- What was the most difficult part of this exercise?
- What was the most important thing you learned from today’s lesson?
- What is still confusing?
Lesson Design
Previous attempts to teach construct validity have been lecture-oriented—the instructor explains construct validity and the logic behind validity studies. After fairly complete exposure to the topic, students still do not grasp the concept. This is particularly obvious when students try to explain how to determine a test’s validity. For example, one way to determine a test’s validity is to administer it to different groups of people and then infer from their performance the level of the test’s validity. Scores for depression should be “higher” for a group of clinically depressed patients than for a comparable group of non-clinically depressed individuals. If so, we conclude the test is a valid measure of depression. If the scores between the groups are similar, then the test is not a valid measure of depression. Students have difficulty explaining how to use test scores with different groups to determine the validity of a test.
Rather than just explain the concepts to students, we created a lesson in which students develop a test of depression, and then devise ways to determine its validity. Students first define a construct (i.e., in this case, “depression”). We decided on “depression” because most students are likely to have encountered the concept in other classes and in everyday life. After defining the concept, students develop several items they believe measure the construct, and then propose ways to test the validity of the test. We do not expect them to sail through the activity without difficulty. The lesson is designed so that the instructor can observe where students do have difficulty understanding the concepts. In this way the lesson makes students’ thinking more visible and open to analysis. The instructor can intervene with additional examples, explanation and discussion when needed.
Predictions
Students will work in small groups to develop a paper and pencil test of depression and determine ways to evaluate the construct validity of their test.
Major steps in the lesson:
1. Students read their individual definitions of depression and use these to develop a group definition.
• Expect students to share their definitions and combine or integrate ideas into a group definition.
• Definitions may not be sophisticated or comprehensive given the short time period (10 Minutes).
• Do students recognize they have invented a “construct?”
2. Based on their definition, each group creates test items to measure the characteristics of depression
• Students develop Likert scale items. May be challenging to form good questions in this format.
3. The group proposes research studies to determine the validity of their test
• This is the most difficult part of the lesson. Students need to examine the logic of construct validity studies in order to propose an appropriate study.
• Students will need to connect their prior understanding of validity studies to their test of depression.
• Anticipate that some groups will not understand the logic behind the validity studies. The instructor will monitor groups and intervene if they are struggling. May address the entire class to explain some of the sticking points.
4. The instructor gives the groups some additional tests (measures) and instructs them to decide how
they could use the tests to determine whether their test of depression is valid.
• Students will need to apply prior knowledge of statistical methods and recognize the extent to which scores on the measures would be related to one another—depending upon whether the tests measure or do not measure depression.
5. Class discussion to review some of the groups’ tests and methods use to determine validity and, time
permitting, individual writing exercise
• The instructor uses the end of class to help consolidate students’ understanding of construct validity and the logic of validity studies.
• Students reflect on their own understanding at the end of class—what they got, didn’t get, what’s confusing.
Contact: Bill Cerbin
Previous Logs: Step 1 | Step 2 |
Project Log # 6: Repeating the Process
Carmen Wilson taught the revised lesson on construct validity, Wednesday, March 2, 2005 in Psychological Measurement 451/551. Observers included Rob Dixon, Melanie Cary, Bill Cerbin, Betsy Morgan and Bart Van Voorhis.
CONSTRUCT VALIDITY
Lesson Plan
Previous activities: In previous classes, students developed definitions for depression individually. I then assigned them to groups in which they came up with a group definition of depression and essential characteristics of a depressed person. They then wrote 5 likert-type items to assess depression. We will be completing the next three steps in today’s lesson.
Step 1: 30 to 40 minutes
o Instruct groups to finish writing items and begin to propose research studies to determine if their tests actually measures depression
o Remind them:
Unlimited resources – funding, people from a variety of settings (e.g. clinics, university, whatever), ages, diagnoses, etc.
Think about your definitions of depression, what you know about depression as you think about what results you might expect
o Monitor groups progress; if groups seem to be going well, let them continue
o If groups struggling, off track call groups attention
Pick some examples to review
Step 2: 15 minutes
o Call for groups attention and provide instructions for final method
o Distribute additional measures
o Think about how you might use these tests to provide evidence of the ability of your test to measure depression well
Hint: think about the statistical methods we have covered in class to this point
Step 3: 10 minutes
o After all groups have completed the task, if time, discuss some of the definitions and methods
o Have group members describe as worksheets are displayed on visualizer
o Save 5 minutes at end (if possible) for writing exercise:
1) What was the most difficult part of this exercise?
2) What was the most important thing you learned from today’s lesson?
3) What is still confusing?
Posted by: Bill Cerbin | March 25, 2005 at 08:00 AM