Lesson Topic: Construct Validity
The lesson will focus on the concept of construct validity. Validity is a core concept in psychological measurement and refers to the extent to which a psychological instrument actually measures the qualities or characteristics it is designed to measure. There are several forms of validity and specific techniques to establish the validity of a psychological instrument. Psychology undergraduates need a well developed understanding of validity in order to evaluate the quality of psychological instruments. Students who go on to practice or do research in psychology need a deep understanding of validity in order to evaluate the quality of psychological instruments or develop credible instruments. Moreover, understanding of validity is helpful for being able to evaluate claims made in the media or work settings about human behavior and qualities.
Carmen Wilson teaches the department’s course in psychological measurement. She has observed that a large percentage of students have difficulty with the concept of construct validity. Currently, she teaches validity as a three part model that includes content validity, criterion validity, and construct validity. In general, content validity addresses the question “Do the items on the test adequately represent the domain of all possible items?” Criterion validity addresses the question “Do scores on the test predict something about a person’s behavior, score on some other measure, or some other non-test behavior?” Construct validity, often conceived of as an umbrella concept including the other two types of validity, addresses the question “Does this test measure what it claims to measure?” Construct validity is theory dependent. Given the theory about some construct (i.e. a hypothetical trait such as depression or intelligence), we can make predictions about how people with high or low levels of the construct might behave. For example, given current theory of the biology of depression, we can predict that depressed individuals who take anti-depressant medications will experience lower levels of depressed after 2 to 4 weeks of taking the medications. The logic of construct validity follows the theory. If a test actually measures depression, depressed individuals should receive higher scores on the test before they take medication than they do after they have been taking the medication for 2 to 4 weeks.
Students have had trouble understanding that logic. Another process for examining construct validity is to look at group differences. For example, if a test does a good job of measuring depression, a group of individuals who are diagnosed as depressed should score higher on the test than a group of individuals who are not diagnosed as depressed. When students are asked this question on an exam, many students become confused, and focus on demographic group differences while failing to address the logic of how different groups should score differently on the test. The actual exam question and some examples of student confusion are listed below.
Test question: One of the processes used to examine construct validity is examining group differences. Explain the logic behind this process. (2 pts)Some classic problems:
“The environment in which you are raised in has a huge effect on your performance. Sometime students that have lower socioeconomic status tend to not do as well in academics because it may be due to higher stress levels. Ethnicities also have to same effect sometimes. Your ethnicity may have biological as well as environmental differences. How we are perceived sometime has an effect on how we do in many aspect of society”“The logic behind this is that people are very different from one another. Going from culture to culture or even looking at gender, you can find many things that are different. We need to take this into consideration when working questions on a test because someone may take a question differently than another person.”
We decided that a research lesson on construct validity would be very valuable given that so many students do not understand this important concept.
Learning Goal
The aim of the lesson is to develop students’ understanding of construct validity, and in particular their understanding of the logic behind the evaluation of construct validity. As a result of the lesson students should be able to explain several ways to assess whether a measure has acceptable construct validity, and demonstrate knowledge of the logic behind these methods.
Overarching Purposes
· To develop students’ ability to analyze and evaluate instruments that measure psychological characteristics.
· To develop “informed skepticism” toward popular claims about human characteristics depicted in the media, in work settings and everyday life.
· To appreciate the differences between popular conceptions of human qualities and those based on careful measurement.
Contact: Bill Cerbin
Previous Posts: Step 1 Log
Carmen Wilson taught the revised lesson on construct validity, Wednesday, March 2 in Psychological Measurement 451/551. Observers included Rob Dixon, Melanie Cary, Bill Cerbin, Betsy Morgan and Bart Van Voorhis.
CONSTRUCT VALIDITY
Introduction to the problem
Construct validity, addresses the question “Does this test measure what it claims to measure?” Construct validity is theory dependent. Given the theory about some construct (i.e. a hypothetical trait such as depression or intelligence), we can make predictions about how people with high or low levels of the construct might behave. For example, given current theory of the biology of depression, we can predict that depressed individuals who take anti-depressant medications will experience lower levels of depressed after 2 to 4 weeks of taking the medications. The logic of construct validity follows the theory. If a test actually measures depression, depressed individuals should receive higher scores on the test before they take medication than they do after they have been taking the medication for 2 to 4 weeks.
Students have had trouble understanding that logic. Another process for examining construct validity is to look at group differences. For example, if a test does a good job of measuring depression, a group of individuals who are diagnosed as depressed should score higher on the test than a group of individuals who are not diagnosed as depressed. When students are asked this question on an exam, many students become confused, and focus on demographic group differences while failing to address the logic of how different groups should score differently on the test. The actual exam question and some examples of student confusion are listed below.
Test question: One of the processes used to examine construct validity is examining group differences. Explain the logic behind this process. (2 pts)
Some classic problems:
“The environment in which you are raised in has a huge effect on your performance. Sometime students that have lower socioeconomic status tend to not do as well in academics because it may be due to higher stress levels. Ethnicities also have to same effect sometimes. Your ethnicity may have biological as well as environmental differences. How we are perceived sometime has an effect on how we do in many aspect of society”
“The logic behind this is that people are very different from one another. Going from culture to culture or even looking at gender, you can find many things that are different. We need to take this into consideration when working questions on a test because someone may take a question differently than another person.”
The following lesson was designed to help students discover and better understand (hopefully!) the logic of construct validity.
Goals of the Lesson
Short-term goals/objectives:
1. To develop students’ understanding of construct validity
2. To develop students’ understanding of the logic behind the evaluation of construct validity
3. As a result of the lesson students should be able to explain several ways to assess whether a measure has acceptable construct validity, and demonstrate knowledge of the logic behind these methods.
Long-term goals:
1. To develop students’ ability to analyze and evaluate instruments that measure psychological characteristics.
2. To develop “informed skepticism” toward popular claims about human characteristics depicted in the media, in work settings and everyday life.
3. To appreciate the differences between popular conceptions of human qualities and those based on careful measurement.
Posted by: Bill Cerbin | March 25, 2005 at 07:52 AM