Methodology CHAPTER OUTLINE

Transcription

1 Methodology 2 CHAPTER OUTLINE LEARNING OBJECTIVES INTRODUCTION SOME FUNDAMENTALS Research methods ad statistics Carryig out quality research The role of theory i psychology DESIGNING EXPERIMENTS IN PSYCHOLOGY What ca we measure? A rudow o research methods Experimet versus survey Which is the best method to use? Decidig what to maipulate Decidig what to measure Differet ways of measurig Producig trustworthy results STATISTICS IN PSYCHOLOGY Samples ad populatios sortig out the jargo Describig umerical results How ca we cofidetly geeralize our results? Fidig out if our results are remarkable Judgig whether two variables are related Uderstadig correlatio FINAL THOUGHTS SUMMARY REVISION QUESTIONS FURTHER READING

2 By the ed of this chapter you should appreciate that: Learig Objectives scietific psychologists follow strict methods whe they coduct research; there is a sigificat differece betwee reliability ad validity; there are differet types of psychological data (e.g. behavioural, self-report ad physiological); statistics are vital to psychological research; experimets help us to tackle the questio of causatio; there is a importat differece betwee causatio ad correlatio; ethical research practice is essetial. INTRODUCTION Psychology is the scietific study of behaviour. Psychology is the sciece of the mid. Psychology is a approach to uderstadig behaviour that uses scietific methods. If you study psychology log eough you are likely to hear statemets like these. Why are words like sciece ad method so importat to psychologists? Why do they cotiually talk about ways of doig thigs, rather tha just doig them? Surely ayoe ca be a psychologist just by beig iterested i the mid or i behaviour? The aswers to these questios are actually pretty simple. Whether they admit it or ot, everybody is iterested i the mid ad behaviour. Ad just about everybody believes that he or she is a expert whe it comes to uderstadig their ow metal life ad thoughts. This is uderstadable. After all, eve if all the experts who have cotributed to this book were watchig you read this chapter, for all their expertise, they would ot kow as well as you do what is goig through your mid. But beig iterested i the mid or claimig that you are a expert is ot eough to make you a psychologist. What is more, there are lots of other experts, such as philosophers ad athropologists, who are iterested i the study of mid ad behaviour. What sets psychologists apart from these other experts is their traiig i psychological methods. May of these methods also appear i other scietific disciplies, partly because psychological methods are derived from geeral scietific methods. Not all psychologists agree etirely with all the methods, but they all uderstad them ad kow how to use the oes relevat to their ow work. It is this commo traiig that, more tha aythig else, makes us psychologists. Of course, psychologists have methods for doig may differet thigs. The methods we discuss i this chapter are those used for doig research. I other words, they are methods for fidig out the aswers to questios about the mid ad behaviour. You kow from your ow experiece that differet people ofte come up with differet aswers

3 26 Methodology to the same questio. What is the tallest moutai i the world? You might say Mout Everest, but your aswer is oly correct if you measure the height from sea level, as we geerally do. If you measure from the cetre of the Earth, the some moutais i South America are actually taller (because the Earth is ot quite roud). So the aswer to the questio depeds o how you iterpret it. This does ot ecessarily mea there is o such thig as truth or that everythig is arbitrary, but simply that the methods used to reach a coclusio eed to be uderstood fully. It ca be much harder to aswer iterestig psychological questios tha to measure the height of a moutai. There are plety of psychological questios that have bee asked may times ad have ot bee defiitively aswered. For this reaso, it is all the more importat that we uderstad the ways that differet psychologists try to aswer the questios they are iterested i. The, if we fid a uusual aswer, we ca decide whether it is i some sese worth takig seriously if we first uderstad the methods that have bee used to reach it. I the ieteeth cetury, before the sciece of psychology was established, a lot of people iterested i the mid ad behaviour studied phreology. Phreologists believed that behaviour could be uderstood by studyig the bumps o people s heads. A particular bump i a particular place was supposed to be associated with particular types of behaviour. We ow kow that this is osese. Nosese or ot, if the last hudred or so years of research i experimetal psychology had ot take place, we would have o systematic way of kowig that this is osese (see chapter 14). To this day people put forward bad, sometimes harmful, ideas related to importat tasks such as treatig psychological disorders, assessig employees ability to do a job, or teachig childre to read. Psychological methods allow us to assess these ideas. This meas that we ca move beyod just arguig about whether a idea is good, ad evaluate the evidece with referece to some shared ad pre-determied criteria. This is a vast topic. Eve if every page of this massive text were devoted to research methods ad statistics, most psychologists would agree that there was still more material that you eeded to absorb to become a expert. The aim of this chapter is to provide some sigposts that will help to guide you. It will ot eable you to deal with every statistical ad methodological questio that you might face, but you will lear how to develop ad set about aswerig your ow questios. SOME FUNDAMENTALS RESEARCH METHODS AND STATISTICS We study research methods ad statistics i order to beefit from the sciece of psychology to qualify as a psychologist, to use psychological kowledge i other fields, or simply to apply psychology to issues that crop up i everyday life. Imagie that your employer requires you to sit a itelligece test before you ca be cosidered for higher duties i your orgaizatio. You sit the test ad are told that your IQ score is 110. What does this mea? Your immediate future may deped o this umber, but you ca oly uderstad it if you kow somethig about itelligece testig (see chapter 13) ad stadard scores. Similarly, ewspapers ad other sources are full of reports relatig to research o psychological issues. May draw alarmig coclusios ofte because they mistakely assume that correlatio is the same as causatio (see below). A good kowledge of psychological research methods allows you to avoid makig the mistakes that jouralists, politicias ad may others make because they lack the ecessary scietific uderstadig. Aother cosideratio is that, i most coutries, i order to become a psychologist you eed to be ot just a iformed cosumer of psychological research but also a producer of it. I other words, you eed to coduct a piece (or several pieces) of research. Imagie you were employed by a school to determie whether its studets are more or less itelliget tha studets at aother school, or i a average school. You would ot be able to aswer this questio properly without coductig a wellplaed piece of psychological research. Eve if you do ot go o to further study, the ability to coduct, aalyse ad evaluate psychological research is a very marketable skill that is cetral to a large umber of occupatios, such as marketig, maagemet or policy makig. Fially, the itellectual challeges explored i this chapter ca be stimulatig ad iterestig i themselves. They are ot icosiderable, but if you ca master them you will be better equipped to uderstad psychology as a whole. This is ot because

4 Some Fudametals 27 studyig methodology ad statistics is a ed itself (though it ca be), but because it is a tool that allows you to get more (persoally, itellectually ad scietifically) from doig psychology. CARRYING OUT QUALITY RESEARCH Psychological research eables us to fid out more about huma behaviour ad the metal processes that uderpi it. We also eed to be sure that our aswers are correct. Suppose we are iterested i whether absece makes the heart grow foder. Is it eough simply to look aroud, make iformal observatios ad come to a coclusio we feel comfortable with? I oe sese it is, ad, as aturally iquisitive people, we do this sort of thig all the time as a meas of formig our ow opiios. But this approach ievitably leads differet people to differet coclusios because we each focus o differet iformatio ad have differet experieces, differet agedas. So some people thik absece makes the heart grow foder while others thik the very opposite, that absece leads the heart to wader. To kow which is correct, whe each is correct ad, scietific method a procedure for more importatly, why, we acquirig ad evaluatig kowledge eed to act as scietists, ot through systematic observatio or lay-scietists. Usig the scietific method differetiates experimetatio psychology from other disciplies that address similar questios. The scietific method is a set of procedures for acquirig ad testig kowledge through systematic observatio or experimetatio. Reliability ad validity The most prized qualities of reliability the extet to which a give psychological research are fidig will be cosistetly reproduced reliability ad validity. Put o other occasios simply, reliability relates to our cofidece that a give fidig ca be replicated validity the extet to which a give ad is ot just a freak or study ivestigates what it purports to chace occurrece. Reliability i psychological research ivestigate has much the same meaig i relatio to a car. A reliable car is oe that early always works. A reliable fidig ca early always be reproduced. Validity relates to our cofidece that a give fidig shows what we believe it to show. A valid car is a geuie car (a car that does what it is meat to do). A valid fidig is a geuie fidig (a fidig that is what it purports to be i.e. oe that ehaces your uderstadig i the maer idicated). Imagie we carry out a study i which we sed someoe to a exotic overseas locatio o a umber of occasios to see whether this makes them thik more favourably of their parter. Let us assume that it does. If it does so repeatedly, the it is a reliable fidig. However, the study does ot ecessarily show that absece makes the heart grow foder, but may istead demostrate that exotic holidays make people feel better about their lives i a geeral way icludig about their parters. The fidig is therefore almost certaily ot valid i a specific sese. Disputed validity is oe of the most commo ad thory problems i psychological research. Disputes arise whe fidigs which purport to show oe thig are reiterpreted to suggest that they actually show somethig completely differet. The fact that the validity of research is ofte questioed is o bad thig. Ideed, this form of aalysis ad debate is cetral to psychological research ad to oe s skill as a researcher. Other qualities of good research As well as beig valid ad reliable, psychological research eeds to be public, cumulative ad parsimoious. To become public, research must be published i a reputable scholarly joural. Sometimes, though rarely, it is traslated ito popular writig, as was the work of Freud, Pavlov, Piaget ad Milgram. The likelihood of a piece of psychological research beig adopted for popular publicatio ca deped o such thigs as topicality, shock value or political treds, ad its impact may be trasitory. I cotrast, the criteria for publicatio i scietific jourals are much more clearly laid out, ad they provide a edurig record of the key fidigs that emerge from a particular piece (or programme) of research. Cumulative research builds o ad exteds existig kowledge ad theory. It is ot eough just to collect iformatio i a haphazard or radom fashio. Istead, research should build o previous isights i a give area. Newto expressed this idea clearly whe he observed: if I have bee able to see further tha others it is because I have stood o the shoulders of giats. Geerally speakig, a piece of psychological research does ot have value i isolatio, but by virtue of extedig or challegig other work i the field. The cumulative ature of research is ofte revealed through literature reviews. These are research papers (ormally published i reputable scietific jourals) that discuss the results of multiple studies by differet researchers. I some cases these reviews ivolve statistical aalyses combiig the results of may studies. This process is called meta-aalysis. meta-aalysis a quatitative method for combiig results across a umber of studies by first covertig the fidigs of each study ito a metric for compariso Parsimoious research develops explaatios of fidigs that are as simple, ecoomical ad efficiet as possible. I explaiig the results i a give field, psychologists therefore attempt to accout for as may differet fidigs as possible usig the smallest umber of priciples. For example, it may be that perso A performs better tha perso B o a test of memory because A was more alert as a cosequece of beig tested at a differet time of day. Or A might have igested a psychoactive aget before testig took place, whereas B had ot. By cotrollig for the possible iflueces of time of day, igested substaces ad so o, we are left with the most parsimoious explaatio for why A ad B differ i their level of memory performace.

5 28 Methodology THE ROLE OF THEORY IN PSYCHOLOGY Sciece does ot progress simply through the accumulatio of idepedet facts. These facts have to be itegrated i terms of theoretical explaatios (theories). Theories are statemets of why, ot just what. They are capable of: 1. accoutig for multiple facts, ad 2. predictig what might happe i ovel situatios. The purpose of most psychological research is to test theory a coheret framework used to make sese of, ad itegrate, a umber such predictios i the form of empirical fidigs of hypotheses i.e. statemets of cause ad effect that are derived from a give theory ad tested by research. So theories geerally precede experimetatio, ot vice versa. For example, the statemet that absece makes the heart grow foder does ot provide a theoretical framework, but the followig statemet is distictly more theory-based: separatio from a object causes us to exaggerate a object s qualities (whether good or bad) because memory distorts reality. This is because this statemet attempts to explai ad ot just describe the relatioship betwee separatio ad emotio. Moreover, havig made this statemet, we ca test it by geeratig hypotheses ad hypothesis a statemet about the causal relatioship betwee particular pheomea (i.e. A causes B), usually derived from a particular theoretical framework, which is desiged to be tested via research ivestigatio doig appropriate research. Oe hypothesis might be that people with memory disorders will make less extreme judgemets of abset loved oes tha people without such disorders. DESIGNING EXPERIMENTS IN PSYCHOLOGY WHAT CAN WE MEASURE? Somethig that differetiates psychology from other scieces is that the thigs i which we are iterested metal states ad processes ca ever be directly observed or measured. You caot touch or see a mood, a thought, a dispositio, a memory or a attitude. You ca oly observe thigs that are associated with these pheomea. While this problem does occur i other scieces (such as astroomy), it ca ofte be overcome through techological developmet (e.g. a better telescope). Psychology has made sigificat advaces too (e.g. measurig ski coductace ad brai blood flow), but these techiques still oly allow psychologists to study the outcomes of metal activity, or thigs that are associated with it ever the activity itself. Psychologists have developed three mai types of measure to help them examie metal processes ad states: 1 Behavioural measures These ivolve observatio of particular forms of behaviour i order to make ifereces about the psychological pheomea that caused or cotributed to them. For example, developmetal psychologists (see chapter 9) might observe which toys are approached or avoided by childre i a play situatio. O the basis of such observatios, they might plausibly ifer that decisios to approach a toy are determied by the toy s colourfuless. 2 Self-report measures These ivolve askig people about their thoughts, feeligs or reactio to a particular questio. Provided that it is possible for the participats to reflect cosciously o the relevat thoughts or behaviours, their resposes ca be used either to supplemet other behavioural measures or as data i themselves. So a researcher could ask a six-year-old (but clearly ot a six-moth-old) Which toys do you like? or Did you pick that toy because it was brightly coloured? 3 Physiological measures These ivolve measurig thigs that are believed to be associated with particular forms of metal activity. For example, heart rate or galvaic ski respose (GSR a measure of the electrical coductivity of the ski) ca serve as measures of axiety or arousal. I our developmetal example, researchers might look at childre s heart rate to see whether they become more excited whe particular toys are preseted or take away. Decisios about which of the above measures to use will be dictated by a umber of factors. May of these are practical ad will be liked to other methodological choices. For example, self-report measures are relatively cheap ad easy to admiister, ad so led themselves to survey-based research examiig large umbers of people i aturalistic settigs. O the other had, physiological measures ca be difficult ad expesive to obtai, so they are ormally used oly i experimetal research with very few participats. However, decisios about which measures to use are guided as much by the particular questio a researcher wats to address as by practical cosideratios. A RUNDOWN ON RESEARCH METHODS Psychological research ivolves four mai methods: the (true) experimetal method, the quasi-experimetal method, the survey method (sometimes experimetal method a research method i which oe or more idepedet variables are systematically maipulated ad all other potetially ifluetial variables are cotrolled (i.e. kept costat), i order to assess the impact of maipulated (idepedet) variables o relevat outcome (depedet) variables quasi-experimetal method embodies the same features as the experimetal method but does ot ivolve the radom assigmet of participats to experimetal coditios survey method the systematic collectio of iformatio about differet variables i order to ivestigate the relatioship betwee them

6 Desigig Experimets i Psychology 29 case study method research method that ivolves a sigle participat or small group of participats who are typically studied quite itesively called the correlatioal method), ad the case study method. The experimetal method Oe very commo research maipulatio the process of systematically varyig a idepedet variable method is to maipulate oe or more variables ad to across differet experimetal coditios examie the effect of this (sometimes referred to as the experimetal treatmet or itervetio) maipulatio o a outcome variable. To do this, the researcher examies participats resposes i the presece ad the absece of the experimetal cotrol the method of esurig that the groups beig studied maipulatio. Experimetal are the same except for the maipulatio or treatmet uder ivestigatio differet situatios idetical cotrol is used to make the i every respect except for the presece or absece of the maipulatio. Experimets ca ivolve differet people i each situatio or the same people i differet situatios. People who take part i experimets are called participats, but if you read older research papers they are geerally referred to as subjects. Here is a example. To test the effect of a ew traiig method (a maipulatio) o memory, we might take 100 people ad expose half of them to the experimetal group participats i a experimet who are exposed to a particular level of a relevat maipulatio or treatmet (as distict from a cotrol group) treatmet the experimetal maipulatio of the idepedet variable cotrol group participats i a experimet who are ot subjected to the treatmet of iterest (as distict from the experimetal group) coditio a situatio i a research study i which participats are all treated the same way betwee-subjects desig a research study ivolvig a systematic maipulatio of a idepedet variable with differet participats beig exposed to differet levels of that variable ew method. For reasos we will discuss i more detail below, we would assig participats to the two groups o a radom basis (e.g. by the toss of a coi). We will call the first group the experimetal group, as it is subjected to a relevat experimetal treatmet. The other half of our participats would ot be exposed to the ew traiig method. As they receive o experimetal treatmet, they are referred to as a cotrol group (also discussed i more detail below). After admiisterig the treatmet, we would measure the performace of the two groups o a memory task ad the compare the results. The various levels of treatmet i a experimet (icludig the cotrol) are referred to as coditios. This experimet has two coditios ad a betwee-subjects desig (because the desig ivolves makig comparisos betwee differet participats i differet coditios). Note, however, that the same questio could also have bee addressed i a withisubjects desig, which would ivolve comparig the memory performace of the same people with ad without the ew traiig method. The two basic desigs have differet stregths ad weakesses, which we will discuss below i relatio to issues of experimetal cotrol. The differet coditios i the experimet make up the idepedet variable (or IV), sometimes called the treatmet variable. A variable is simply somethig that chages or varies (is ot costat). I true experimets, the idepedet variable is systematically maipulated or varied by the experimeter. Experimets ca (ad typically do) have more tha oe idepedet variable. Experimets also ivolve at least oe depedet variable (or DV). This is a outcome or measuremet variable, ad it is this variable that the experimeters are iterested withi-subjects desig a research desig i which the same participats are exposed to differet levels of the idepedet variable idepedet variable the treatmet variable maipulated i a experimet, or the causal variable believed to be resposible for particular effects or outcomes depedet variable the variable i which a researcher is iterested i moitorig effects or outcomes i observig ad which provides them with data. I our last example, the depedet variable is the level of memory performace. Use the iitial letter d to remember the lik betwee the depedet variable ad the data it provides. Cotrol is the basis of experimetal desig. It ivolves makig differet coditios idetical i every respect except the treatmet (i.e. the idepedet variable). I a betwee-subjects experimet, this is achieved by a process of radom assigmet of participats to the differet coditios. For example, people should be assiged at radom (e.g. radom assigmet the process of o the basis of coi tossig), assigig participats to study coditios o a strictly usystematic basis rather tha puttig, say, the first 50 people i oe coditio ad the secod 50 i aother. This practice rules out the possibility that there are systematic differeces i, say, itelligece, persoality or age betwee the groups. If there is a differece i results obtaied from measurig the depedet variable for each group, ad we have equated the groups i every respect by meas of radom assigmet, we ca ifer that the differece must be due to our maipulatio of the idepedet variable. The quasi-experimetal method I quasi-experimetal studies the idepedet variable is ot (or caot be) maipulated as such, ad so assigmet to experimetal groups caot be radom. The fact that o maipulatio

7 30 Methodology Pioeer Doald Thomas Campbell ( ) traied as a social psychologist. He was a master methodologist ad is best kow for devisig the method of quasi-experimetatio, a statistics-based approach that allows replicatio of the effects of true radomizatio, which is ofte impossible i the study of huma behaviour. Campbell also supported use of qualitative methods, accordig to the goals ad cotext of the study. He promoted the cocept of triagulatio that every method has its limitatios, ad multiple methods are usually eeded to tackle importat research questios. occurs iterferes dramatically with our ability to make coclusive causal ifereces. Examples of idepedet variables that caot be maipulated by a experimeter iclude geder ad age. Obviously experimeters caot chage the geder or age of participats, but they ca compare the resposes of groups of people with differet ages or of differet geders. Compared to the experimetal method, there is o real cotrol over the idepedet variable, so we caot coclude that it is ecessarily resposible for ay chage i the depedet variable. O this basis, as we will see, the quasi-experimetal method actually has more i commo with survey methodology tha with the experimetal method. It has all the weakesses of the experimetal method, but it lacks the mai stregth. I practice, it is ofte coducted i cojuctio with the experimetal method. For example, i our learig study we might compare the effect of the ew traiig method o both me ad wome. The survey (or correlatioal) method The survey method is commoly used to idetify the aturally occurrig patterig of variables i the real world rather tha to explai those patters (though ofte people wat to put a explaatory gloss o them). So to examie whether absece makes the heart grow foder we could coduct a survey to see if people who are separated from their parters because of travellig away from home (group A) say more positive thigs about their parters tha people who ever travel away from home without their parters (group B). This might be a iterestig exercise, but the validity of ay causal statemets made o the basis of such fidigs would be very limited. For example, if we foud from our survey that group A said more positive thigs about their parters whe they were travellig tha group B, it would be impossible to demostrate coclusively that absece was the cause of the differece betwee groups A ad B. I other words, while our survey could show us that absece is associated with a foder heart, it could ot coclusively show that absece actually causes the heart to grow foder. It is quite possible (odd as it may soud) that the sorts of people who travel away from home without their parters are simply those that like their parters more (so fodess makes the heart go abset). Or perhaps both fodess ad absece are caused by somethig else for example, social class (i.e. beig wealthy makes people both fod ad abset). I large part, the, surveys rely o methodologies that idetify relatioships betwee variables but do ot allow us to make coclusive causal ifereces. The case study method Most of the above methods are used for studies ivolvig large umbers of participats. But what if oly a few are available? How, for example, would you do research if you were iterested i the readig difficulties of people with particular forms of brai damage? To ivestigate questios like this, researchers ofte resort to the case study method, which ivolves itesive aalysis of a very small sample. This has particular problems (ofte with reliability), but some of the most famous studies i psychology have used this method i particular the work of Freud (see chapter 14). Takig a qualitative approach Whe researchers report ad commet o behaviour, without attemptig to quatify it, they are usig a qualitative research method. This ivolves attempts to uderstad behaviour by doig more tha merely covertig evidece ito umbers. Qualitative methods ca iclude codig, groupig ad collectig observatios without assigig actual umbers to the observatio. So a qualitative aalysis of the speed of aimals might result i the statemet that the cheetah is a fast lad aimal, ad a quatitative aalysis might ivolve comparig the maximum speed of aimals over (say) 20 metres. To take a example of huma behaviour, you probably take a qualitative approach to the friedliess of the people you meet. I other words, you probably judge people as relatively friedly or ufriedly, but you would be ulikely to come up with a umber that expresses their friedliess quotiet. Qualitative techiques are sometimes used i the iitial stages of quatitative research programmes to complemet the quatitative techiques, but they are also used by psychologists who challege covetioal approaches to psychological research. This may be because they believe that the covetioal methods are iadequate for addressig the richess ad complexity of huma behaviour. I tur, may maistream psychologists are critical of qualitative methods. (For further discussio of qualitative methods, see Haslam & McGarty (2003).) EXPERIMENT VERSUS SURVEY Oe commo, but mistake, belief is that the differece betwee surveys ad experimets is a questio of locatio, with surveys beig coducted i the commuity ad experimets i the laboratory. This is ofte the case, but ot always. Experimets ca be coducted outside laboratories, ad surveys ca be coducted i them.

8 Desigig Experimets i Psychology 31 The mai differeces betwee experimets ad surveys relate to the sorts of questios that each ca aswer. As we suggested earlier, experimets ted to be cocered with establishig causal relatioships betwee variables, ad they achieve this by radomly assigig participats to differet treatmet coditios. Surveys, o the other had, ted to be cocered with measurig aturally occurrig ad edurig relatioships betwee variables. Researchers who use surveys usually wat to geeralize from the sample data they obtai to a wider populatio. They do this by usig the sample to estimate the characteristics of the populatio they are iterested i. Why choose to carry out a survey rather tha a experimet? Two reasos: sometimes we are oly iterested i observig relatioships, ad sometimes maipulatios simply are ot possible. This reasoig is ot restricted to psychology. Astroomers or geologists rarely coduct experimets, simply because it is ofte impossible to maipulate the idepedet variables of iterest (e.g. the positio of certai stars or the gravitatioal force of a plaet). Istead they rely largely o the same logic of cotrolled observatio that uderpis psychological surveys. But this does ot mea that astroomy or geology are uscietific. Surveys ca also allow researchers to elimiate some causal liks. If there is o relatioship (at least i the survey eviromet) betwee variables, this allows us to coclude that oe does ot cause the other. For example, if o relatioship is foud betwee age ad itelligece, the it is impossible for itelligece to cause age, or vice versa (bearig i mid that a relatioship could be cocealed by a third, or backgroud, variable). WHICH IS THE BEST METHOD TO USE? This is a very complex issue ad depeds o may factors, ot least practical oes icludig the amout of time, moey ad expertise that a researcher has. However, as a geeral priciple, it is worth emphasisig that o oe method is uiversally superior. Part of ay research psychologist s role is to make judgemets about the appropriateess of a method for ivestigatig the issues at had. Beig a good researcher is ot a questio of whether you do experimets or surveys: it is more a matter of whe ad how you do them. I view of the potetial limitatios of ay oe method, may researchers cosider usig multiple research methods to explore the same issue i may differet ways. This is the process of triagulatio. If cosistet results are obtaied from a variety of differet methods (perhaps from a quatitative experimet, a survey ad qualitative case studies), this will ted to justify greater cofidece i the fidigs. For this reaso, the eed to make methodological choices should be see as a asset for researchers, rather tha a basis for argumets about who has the best methods. The challege researchers face is to exploit that asset appropriately. DECIDING WHAT TO MANIPULATE I selectig a idepedet variable for ay piece of research, we must first decide what we are iterested i. For example, we might be iterested i whether attributioal style (the way people explai evets) affects people s resposes to failure. We might hypothesize that people who ted to blame themselves for failure (i.e. those who iteralize failure) are more likely to become depressed tha people who blame their failure o other thigs (i.e. who exteralize failure). So the cetral theoretical variable the focus of our iterest is the participats attributioal style. But, how ca we maipulate this for the purposes of our experimet? Clearly we caot ope up people s heads ad tur a dial that says attributioal style to maximum or miimum. To get aroud such obstacles, psychologists usually maipulate the theoretical variable idirectly. They do this by idetifyig a idepedet variable that they believe will have a specific impact upo a give metal process, ad the check that this is the case. I our example, the researchers may expose participats to failure (e.g. i a test) ad the ask some of them to aswer questios like Ca you explai why you did so much worse tha everyoe else? questios that ecourage the participats to reflect o their ow cotributio to their performace (i.e. to iteralize). They may the ask other participats questios like Do you thik the fact that you were ot allowed to revise for the test affected your performace? questios that ecourage them to reflect o the cotributio of other factors to their performace (i.e. to exteralize). To be sure that this maipulatio has had the desired effect o the theoretical variable, the researchers may the wat to perform a maipulatio check. For example, i the case give above, the researchers might measure maipulatio check a procedure that checks the maipulatio of the idepedet variable has bee successful i chagig the causal variable the experimeter wats to maipulate whether the iteralizig questio produces greater agreemet with a measure such as: How much do you thik you were resposible for the test outcome? Note also the sigificat ethical issues relatig to this study. The experimetal maipulatio could have the effect of makig some participats more depressed ideed, that is the hypothesized outcome i the coditio where participats are ecouraged to iteralize their failure. We discuss ethical issues later i this chapter. DECIDING WHAT TO MEASURE As with the selectio of IVs, the selectio of depedet variables is ofte complicated by practical costraits. For example, if we are ivestigatig the impact of alcohol cosumptio o road fatalities, we may maipulate the idepedet variable straightforwardly (by gettig experimetal groups to cosume differet quatities of alcohol). But it would be irresposible (ad illegal) to the get the participats to drive dow a busy street so that we ca cout how may pedestrias they kock dow! To get roud this, we may ask the high alcohol group to cosume oly a few beverages. But there are two problems with this.

9 32 Methodology Everyday Psychology Testig the effectiveess of therapy for depressio I m feelig depressed. Most likely you have heard someoe say this. But such statemets should ot be cofused with cliical depressio, a disorder that produces greater impairmet i everyday fuctioig tha may physical health problems (e.g. hypertesio, arthritis, diabetes; see chapter 15). You have approximately a 15 per cet chace of experiecig cliical depressio i your lifetime. Should you be ufortuate eough to experiece a depressive disorder you would surely wat to get treatmet for it that is effective. Cogitive behaviour therapy ad o-directive cousellig are commo treatmets for people with depressive symptoms (see chapter 16). But are these treatmets ay more effective tha usual geeral practitioer (GP) care? You might tur to psychological research for a aswer. But how ca psychologists evaluate the effectiveess of cogitive behaviour therapy, o-directive cousellig ad usual GP care i treatig depressio? A effective test requires the use of a experimetal desig i which patiets are radomly assiged to treatmet groups. This is ecessary to allow ay effects to be attributed to treatmet type, rather tha ay other variable that might lead a participat to choose oe treatmet over aother. Ward et al. (2000) followed this procedure ad allocated patiets to oe of three treatmet groups: two psychological treatmets (o-directive cousellig ad cogitive behaviour therapy) ad oe cotrol coditio (usual GP care). They measured the patiets level of depressio before treatmet bega, at four moths ad at 12 moths followig the completio of the treatmet. What did they fid? At four moths, patiets i both the psychological treatmet groups (o-directive cousellig ad cogitive behaviour therapy) had sigificatly lower depressio scores tha patiets i the cotrol coditio (usual GP care). There was o sigificat differece betwee the effectiveess of the two psychological treatmets. We ca coclude from this experimetal test (kow as a cliical trial whe treatmets are beig tested; see chapter 16) that the two psychological treatmets for depressio are effective, at least i the short term. Ideally, however, you would wat a treatmet that produces lastig results, especially i light of the fact that depressio teds to be both chroic ad recurret (see chapter 15). But whe Ward et al. examied depressive symptoms at 12 moths followig treatmet there was o sigificat differece betwee ay of the three groups. I this study, the, the psychological treatmets for depressio were show to be effective i the short term but ot i the log term. Happily, there are other studies that documet the loger-term effectiveess of cogitive behaviour therapy as a treatmet for depressio (see chapter 16). Ward, E., Kig, M., Lloyd, M. et al., 2000, Radomised cotrolled trial of o-directive cousellig, cogitive-behaviour therapy, ad usual geeral practitioer care for patiets with depressio. I: Cliical effectiveess, British Medical Joural, 321, First, alcohol may oly affect drivig behaviour whe more tha a few beverages are cosumed. Secod, our depedet variable (umber of pedestrias killed) will ot be sufficietly sesitive to detect the idepedet variable s impact. I other words, we may have good reaso to thik that alcohol could impair drivig performace, but the degree of impairmet may ot (fortuately!) be so profoud as to cause a detectable icrease i the umber of deaths caused. To deal with this, we therefore have to select depedet variables that are both relevat to the outcome we have i mid ad sesitive to the idepedet variable. I the case of drik-drivig, we may look at participats reactio time, because we believe that this is a critical determiat i drivig safety ad is likely to be a sesitive eough variable to detect a impairmet i drivig performace due to alcohol. We ca the desig ad carry out a study i the laboratory, measurig the impact of alcohol cosumptio o reactio time. I our attributioal style example, too, it is ulikely that our maipulatio of the idepedet variable will have a dramatic impact o the participats depressio. So if our depedet variable was the umber of participats who eed to be treated by a cliical psychologist, our experimet is very ulikely to ucover ay effects. To get aroud this problem, we could admiister a depressio ivetory, i which we ask the participats a battery of questios (e.g. Are you self-cofidet?, Do you feel hopeless about the future? ) i order to measure their susceptibility to depressio. We could the test our hypothesis by seeig whether scores o the depressio ivetory revealed a higher susceptibility to depressio amog participats who had bee ecouraged to make iteral attributios. DIFFERENT WAYS OF MEASURING The psychologist S.S. Steves developed a famous distictio betwee forms of data that psychologists ca deal with. The four types he came up with are omial, ordial, iterval ad ratio measures.

10 Statistics i Psychology 33 Pioeer Staley Smith Steves ( ) made sigificat cotributios to several areas of psychology. He was a expert o the psychophysics of hearig ad was iterested i measuremet ad experimetal psychology. Steves set out to redefie psychological measuremet by chagig the perspective from that of ivetig operatios (the physical view) to that of classifyig scales (a mathematical view). He also discovered that methods such as just oticeable differeces, ratig scale categories ad paired comparisos produce oly ordial scales. Steves most oustadig cotributio was his successful argumet that there are differet kids of scales of measuremet, beig the first to defie ad discuss omial, ordial, iterval ad ratio scales. Nomial measures The data collected i this way are i the form of ames, which ca be categorized but caot be compared umerically i ay way. Examples iclude geders, coutries ad persoality types. Ordial measures These ca be raked i some meaigful way. Examples are the placigs obtaied by competitors i a race or a ordered set of categories (e.g. low stress, moderate stress ad high stress). Iterval measures Numerical measures without a true zero poit are called iterval measures, ad caot be used to form ratios. A example is temperature. The zero poit has bee arbitrarily chose to be the freezig poit of water rather tha absolute zero (where there is o temperature), ad it is simply ot true that 40 degrees Celsius is twice as hot as 20 degrees Celsius. Similarly, it would ot make sese to say that someoe who respoded with a 6 o the attributio scale above was twice as much of a exteralizer as someoe who respoded with a 3. Ratio measures Full umerical measures with a true zero poit are ratio measures. Psychologists frequetly assume that scores obtaied from psychological measuremet ca be treated as ratio measures. But this assumptio is ot always justified. PRODUCING TRUSTWORTHY RESULTS Iteral validity We ca be cofidet about the results of psychological research whe the methods are valid. A experimet is said to have iteral validity whe we are cofidet that the results have occurred for the reasos we have hypothesized, ad we ca rule out alterative explaatios of them. These alterative explaatios (or threats to iteral validity) ca ivolve a experimetal cofoud a uiteded maipulatio of a idepedet variable. The risk of cofouds ca be reduced by better experimetal desig. Suppose we coduct a study to look at the effect of crowdig o psychological distress by puttig 50 people i a crowded room ad 50 people i a ope field. Havig foud that the people i the room get more distressed, we may wat to coclude that crowdig causes distress. But the participats body temperature (geerated by havig a lot of people i oe room) may represet a cofoud i the study: it may be the heat, ot the crowdig, that produces the effects o the depedet variable. The experimet could be redesiged to cotrol for the effects of this cofoud by usig air-coditioig to keep the temperature the same i both coditios. A study has a high level of exteral validity whe there are o reasos to doubt that the effects obtaied would occur agai outside the research Exteral validity iteral validity the extet to which the effect of a idepedet (maipulated) variable o a depedet (outcome) variable is iterpreted correctly cofoud a uiteded or accidetal maipulatio of a idepedet variable that threates the validity of a experimet exteral validity the extet to which a research fidig ca be geeralized to other situatios settig. We might, for example, questio a study s exteral validity if participats respoded i a particular way because they kew that they were takig part i a psychological experimet. They might iadvertetly behave i a way that either cofirms or udermies what they believe to be the researcher s hypothesis. I experimets we usually try to deal with this specific potetial problem by ot tellig experimetal participats about the hypotheses that we are ivestigatig util after the experimet has fiished. STATISTICS IN PSYCHOLOGY SAMPLES AND POPULATIONS SORTING OUT THE JARGON You will ofte hear psychologists talkig about samples ad populatios i relatio to statistical aalysis of research. What do they mea by these terms? A populatio is a set of people, thigs or evets that we are iterested i because we wish to draw some coclusio about them. The populatio could cosist of all people, or all people

11 34 Methodology with schizophreia, or all right-haded people, or eve just a sigle perso. A sample is a set selected from the populatio of iterest ad used to make a iferece about the populatio as a whole. This kid of iferece is called a geeralizatio. A sample would ormally be a group of people selected from a larger group, geeralizatio related to the cocept but it could also be a sample of exteral validity, this is the process of of behaviour from oe makig statemets about the geeral perso, or eve a sample of populatio o the basis of research euros from a regio of the brai (see chapter 3). If we wish to geeralize to a populatio, we eed to make sure that the sample is truly represetative of the populatio as a whole. This meas that the sample should be similar to the populatio i terms of relevat characteristics. For example, if we are doig research o the huma visual system, the members of our sample group eed to have eyesight that is similar to the rest of the huma populatio radom sample a sample of participats i which each has the same chace of beig icluded, esured by usig radom participat selectio methods (e.g. drawig lots) (as opposed to beig, for example, oticeably worse). The easiest ad fastest way to achieve this is to draw a radom sample (of a reasoable size) from the populatio. DESCRIBING NUMERICAL RESULTS Two key properties, referred descriptive statistics umerical statemets about the properties of data, to as descriptive statistics, come ito play whe we such as the mea or stadard deviatio describe a set of data or the results of our research. These are the cetral tedecy (what we usually call the average) cetral tedecy measures of the ad the amout of dispersio average (most commoly the mea, or variatio. media ad mode), which tell us what Imagie a choreographer costitutes a typical value selectig a group of dacers for a performace supportig a lead dacer who has already bee cast. The choreographer wats the support- dispersio measures of dispersio (most commoly rage, stadard deviatio ig cast to be pretty much ad variace) describe the distace of the same height as the lead separate records or data poits from dacer ad also pretty much each other the same height as each other. So the choreographer is iterested i the average height (which would eed to be about the same as the lead dacer s height) ad the dispersio, or variatio, i height (which would eed to be close to zero). There are a umber of ways i which the choreographer or the psychologist ca measure cetral tedecy (average) ad dispersio. Measures of cetral tedecy Measures of cetral tedecy give us a typical value for our data. Clearly, typical ca mea differet thigs. It could mea: the average value; the value associated with the most typical perso; or the most commo value. I fact, all three are used by researchers to describe cetral tedecy, givig us the followig measures: The mea is the average value (respose) calculated by summig all the values ad dividig the total by the umber of values. The media is the value with a equal umber of values above ad below it. So, if all values are raked from 1 to N, the media is the ((N + 1)/2)th value if N is odd. If N is eve, the media is the mea of the two middle values. The mode is the value that occurs most frequetly i a give data set. Measures of dispersio We might also wat to describe the typical distace of resposes from oe aother that is, how tightly they are clustered aroud the cetral poit. This is typically established usig oe of two measures. The first ad probably most obvious is the rage of resposes the differece betwee the maximum ad miimum values. But i fact the most commoly used measure of dispersio is stadard deviatio (SD). This is equal to the square root of the sum of the squares of all the differeces (deviatios) betwee each score ad the mea, divided by the umber of scores (i fact, the umber of scores mius oe if we wat a populatio estimate, as we usually do). If this souds complex, do ot be too cocered: scietific calculators allow you to compute stadard deviatios very easily. The square of the stadard deviatio is called the variace. mea the sum of all the scores divided by the total umber of scores media the middle score of a raked array equal to the ((N + 1)/2)th value, where N is the umber of scores i the data set mode the most commoly occurrig score i a set of data stadard deviatio the square root of the sum of the squares of all the differeces (deviatios) betwee each score ad the mea, divided by the umber of scores (or the umber of scores mius 1 for a populatio estimate) variace the mea of the sum of squared differeces betwee a set of scores ad the mea of that set of scores; the square of the stadard deviatio

12 Statistics i Psychology 35 Research close-up 1 A survey o psychiatric disorders The research issue Util the mid 1980s, research ito the prevalece of psychiatric disorders, such as affective (mood) disorders, relied o istitutioal records. Lubi et al. (1988) set out to ivestigate the relatioships betwee affect ad demographic ad physical health variables i a represetative populatio sample. Desig ad procedure The Revised Multiple Affect Adjective Check List (MAACL-R) was admiistered to 1,543 adults throughout the Uited States. This sample was desiged to produce a approximatio of the adult civilia populatio at the time. The MAACL-R provided measures of five traits: axiety, depressio, hostility, positive affect (optimistic mood state) ad sesatio seekig. I additio, participats were iterviewed i order to elicit demographic iformatio ad subjective impressios of physical health. Results ad implicatios The sesatio-seekig scale was ot foud to be iterally reliable ad so was ot used i further aalyses. The researchers aalysed the data primarily usig t-tests, aalysis of variace ad correlatio. Statistical tests showed that females scored sigificatly higher tha males o measures of axiety, depressio ad positive affect. Correlatioal aalyses revealed sigificat relatioships betwee measures of affect ad subjective measures of physical health. Positive affect had a highly sigificat positive correlatio with physical health (r =.30), while axiety (r =.10), depressio (r =.20) ad hostility (r =.09) were all sigificatly egatively correlated with physical health (though the correlatios were relatively small). Note, however, that this study does ot allow us to coclude that feelig healthy causes oe to be happier, less axious, less depressed ad less hostile, or that beig happy (ad ot axious, depressed or hostile) causes oe to be physically healthy. Lubi, B., Zuckerma, M., Breytspraak, L.M., Bull, N.C., Gumbhir, A.K., & Rick, C.M., 1988, Affects, demographic variables, ad health, Joural of Cliical Psychology, 44, Compared to the rage aloe, stadard deviatio tells us a lot about a distributio of scores, particularly if they are ormally distributed a feature we discuss further below. If this is the case, we kow, for example, that about 68 per cet of all values will fall withi 1 SD of the mea, 95 per cet fall withi 2 SDs ad 99 per cet fall withi 3 SDs. For reasos that will become clear i later chapters (e.g. chapter 13), this sort of iformatio is very useful. HOW CAN WE CONFIDENTLY GENERALIZE OUR RESULTS? Although psychologists ofte sped a lot of time studyig the behaviour of samples, most of the time they wat to geeralize their results to say somethig about a whole populatio ofte called the uderlyig populatio. Kowig how te particular people are goig to vote i a electio may be iterestig i itself, but it is eve more iterestig if it tells us who is likely to wi the ext electio. But how ca we make ifereces of this sort cofidetly? By usig iferetial statistics we ca make statemets about uderlyig populatios based o detailed kowledge of the sample we study ad the ature of radom processes. The key iferetial statistics umerical techiques used to estimate the probability that purely radom samplig from a experimetal populatio of iterest ca yield a sample such as the oe obtaied i the research study poit here is that, while radom processes are (as the ame tells us) radom, i the log ru they are highly predictable. Not coviced? Toss a coi. Clearly, there is o way that we ca cofidetly predict whether it is goig to come dow heads or tails. But if we were to toss the coi fifty times, we could predict, reasoably accurately, that we would get aroud twety-five heads. The more tosses we make, the more certai we ca be that aroud about 50 per cet of the tosses will come up heads (ad it is this certaity that makes the busiess of ruig casios very profitable).