Reliability and Validity

Transcription

1 Reliability and Validity Why Are They Important? Check out our opening graphics. In a nutshell, do you want that car? It's not reliable. Would you recommend that car magazine (Auto Tester Weakly) to a friend? Its recommendations are not valid. You will face many important decisions that will affect people s lives if you become a professional psychologist. For example:

2 Is this person clinically depressed? Does this child really have a learning disability? Is this treatment or drug really going to help this person? You will want to choose measures and you will want to evaluate studies based upon whether or not they are reliable and valid. Your knowledge of these two concepts is one of the most important tools you ll have. Besides, if you are going to remain a psychology major, then believe me, you are going to hear about reliability and validity until the cows come home. So buckle in and resolve to get these two concepts down pat. They are two very different ideas, so make sure you can tell the difference between them. First, understand this: psychologists measure things they believe exist in people's heads. Pretty crazy. Think about it: Have you ever seen a person's self-esteem? Can you get out a ruler and measure someone's intelligence? Are you extroverted? Then where exactly is your "extroversion" located? About three inches in from your right eye and then 1 inch over to the left? Moral: psychologists measure things that we cannot see, but we believe them to exist inside your head. We had better be able to support that belief. To do it, we rely on reliability and validity. Let's take a look. Reliability What would you do with a bathroom scale that gave you a different weight every time you stood on it? You'd throw it out. In the same way, if you are going to have any faith in a psychological measure (like intelligence or extroversion), then you at least have to get the same score (or at least something close) each time you give the test to someone. That's reliability. You gotta have it or no one will believe that you know what you are talking about. What about all the items on a standardized test that are supposed to measure some construct like mathematical ability? You would think that all the items would tend to agree with each other. I mean that one subset of items shouldn't say you stink in math while another subset says that you are great. This is another form of reliability. Ever take a survey over the phone and they seem to ask you the same question twice. They are checking to see if you are reliable.

3 Moral: When you think about reliability, think CONSISTENCY. If the psychological concept you are interested in really does exist inside people s heads, then you should always find it when you look for it. If you do always find it, that is, if your subjects always respond consistently, then people will start to believe that you really have something there. Or in other words, your car should start each day and not catch fire. There are different kinds of reliability. Here s a graphic to help you organize and remember these important ideas: Statistical Nuances The actual statistics used to test reliability can be quite complex. However, the ideas are simple and really just forms of correlation and regression. We'll just give you a small taste of the procedures. Say you have a test that measures a personality trait. You would like for all the items to give you consistent information about the trait. How could you do that? Here's a clever idea - let's take half the items, compute your score and take the other half of the items and compute a separate score for each. If you found a high Pearson's correlation coefficient between these split halves then it would look like the

4 two parts of the test agree with each other. The whole test would seem to have good internal consistency or reliability. Validity Having subjects respond reliably on a measure is a great start, but there is another concept you need to get down really well. That s validity. There are many kinds of validity, but they all refer to whether or not what you are manipulating, or what you are measuring, truly reflects the concept you think it does. Here s a crazy (but true) example: many years ago, people used to believe that if you had a large brain then you were intelligent. Suppose you went around and measured the circumference of your friend's heads because you also believed this theory, (they d know for sure that you're a psychology major now). Is the size of a person s head a reliable measure (Think first!)? The answer is YES. If I measured the size of your head today and then next week, I would get the same number. Therefore, it IS reliable. However, the whole idea is wrong! Because we now know that larger headed people are not necessarily smarter than smaller headed ones, we know that the theory behind the measure is invalid. Moral: When you do research in psychology you have to make sure that you get consistent results that also truly reflect those mysterious concepts that reside in the human mind. To make statements that you can have confidence in means establishing validity. There are several kinds. Let's do a quick review of three common types of validity: Internal, External and Construct. 1. Internal Validity: When you think about internal validity, think INSIDE the experiment. Is your experiment so well designed that when the results are in, you feel confident that you can make truthful and definite statements about what

5 happened in your study? If your study is relatively free of confounds; you will have high confidence in its results. That's internal validity. 2. External Validity: When you think about external validity, think OUTSIDE the experiment. Can your results be generalized to people outside of your study? Whether this is high or low depends on what you are studying and what your subjects are like. Just consider it carefully: will people not in your study react the same way as those in your study? 3. Construct Validity: Think CONCEPT. You are manipulating and measuring many concepts in your study are you really tapping into these concepts? Here's an example: Independent Variable: Suppose you wanted to study the effects of violent TV on aggression in children. Your first step is to decide which TV shows contain "violence". Whatever show you pick, you should first make sure that children perceive what THEY SEE in the show as violence. Is there a difference between cartoon violence and violence that involves real people? You have to think these issues through to make sure that you are truly (validly) manipulating the concept (or "construct" - really the same term) of a violent TV show. Dependent Variable: Suppose you decide (as early researchers did) to measure the "number of times a child hits a bobo doll" as your dependent measure of aggression (a bobo doll is that inflatable doll that bounces back up when you hit it). Now ask yourself: when a child hits a bobo doll, is this because of aggressive tendencies? If the answer is no (because hitting a bobo doll is simply a way of playing with it), then the number of hits on the doll would not accurately measure the construct of "aggression". You want to measure something that truly reflects the construct (concept) that you think is an expression of what is inside people's heads.

6 Here s a graphic to help you organize and remember these important ideas: Bottom Line: Reliability and validity are very easy conceptually but they strike to the heart of Psychology as a useful discipline. If our measurements differ from time 1 to time 2 to time 3 and we measure things that are not useful, then who cares?