Measuring the Ability to Identify One s Own Emotions: The Development and Initial Psychometric Evaluation of a Maximum-Performance Test

Transcription

1 Sesatios Test 1 Measurig the Ability to Idetify Oe s Ow Emotios: The Developmet ad Iitial Psychometric Evaluatio of a Maximum-Performace Test Kimberly A. Barchard ad Jeifer Skeem Uiversity of Nevada, Las Vegas Referece: Barchard, K.A. ad Skeem, J. (2004, April). Measurig the Ability to Idetify Oe s Ow Emotios: The Developmet ad Iitial Psychometric Evaluatio of a Maximum-Performace Test. Poster preseted at the Wester Psychological Associatio Aual Covetio, 2004, Pheoix, Arizoa. Cotact Iformatio: Kim Barchard, Departmet of Psychology, Uiversity of Nevada, Las Vegas, 4505 S. Marylad Parkway, P.O. Box , Las Vegas, NV, , USA, barchard@ulv.evada.edu Abstract Emotioal Itelligece icludes the ability to perceive, uderstad, ad maage your ow emotios ad those of other people. Both self-report ad maximum-performace measures of most of these aspects of Emotioal Itelligece exist. However, there are o maximum-performace measures of the ability to idetify oe s ow emotios. Creatig such a measure poses uique measuremet problems. After all, if a perso claimed that they felt guilty or happy or agry, what criteria ca the outside observer reliably use to determie if the perso is correct? Because of this, it may ot be possible to create a maximum-performace measure that directly taps the ability to idetify oe s ow emotios. However, it may be possible to create a maximum-performace test of a closely allied skill, kowledge of the relatioships betwee emotios ad physical sesatios. The purpose of this study was to create a relatively short test of kowledge of the relatioships betwee emotios ad physical sesatios ad to provide prelimiary evidece of its reliability ad validity. A iitial item pool of 224 emotio-sesatio pairs were admiistered to a sample of 219 udergraduate studets ad each item was correlated with a established measure of Emotioal Itelligece. Based upo these correlatios, five emotios ad 11 sesatios were selected for iclusio o Versio 2 of the Sesatios Test, with 19 of these emotio-sesatios pairs beig scored. Iitial reliability ad validity evidece for the Sesatios Test Versio 2 was the examied. There was evidece of moderate iteral cosistecy ad of coverget validity with other maximum-performace measures of Emotioal Itelligece. The Sesatios Test did ot correlate with self-report measures of Emotioal Itelligece, however, eve whe these self-report scales were desiged to measure precisely the ability to idetify oe s ow emotios. This lack of sigificat positive correlatios highlights the importace of creatig a maximumperformace measure of this skill. Itroductio Emotioal Itelligece is a ew ad excitig cocept for both theoretical ad applied researchers. Emotioal Itelligece ca be broadly defied as the cogitive ability to perceive, uderstad, ad maage your ow emotios ad those of other people. Various approaches to measurig differet aspects of Emotioal Itelligece have bee attempted, ad both self-report (Bar-O, 1997; Salovey, Mayer, Goldma, Turvey, & Palfai, 1995; Schutte et al., 1998; Tett, Wag & Fox, 2003) ad maximum-performace measures (Mayer, Caruso, & Salovey, 2000a, 2000b) of may aspects of Emotioal Itelligece ow exist. However, creatig maximum-performace tests has bee quite difficult, because the experiece of emotios is subjective. This makes it difficult to desig Emotioal Itelligece items where there is clearly oe ad oly oe correct aswer. Measurig the ability to idetify oe s ow emotios has bee particularly difficult. After all, if we ask someoe how they feel, ad they say they feel sad, how would we kow if they are right? Because of this, it may be impossible to create a direct measure of the ability to idetify oe s ow emotios. At preset, oly self-report measures of this skill exist (Bagby, Parker, & Taylor, 1994; Salovey, Mayer, Goldma, Turvey, & Palfai, 1995). However, it may be possible to develop a maximumperformace test that assesses kowledge that is closely allied to this skill: kowledge of the relatioships betwee emotios ad physical sesatios. That was the purpose of this study. The purpose of this study was to create a relatively short maximum-performace test of kowledge of the relatioships betwee emotios ad physical sesatios, based upo the ratioale that this skill is closely related to the ability to idetify oe s ow emotios. This test was called the Sesatios Test. The first versio of this test cotaied 224 emotio-sesatio pairs. The purpose of this study was to twofold: first, to idetify those emotiosesatio pairs that validly assess Emotioal Itelligece, by comparig scores o each item with scores o a well-

2 Sesatios Test 2 validated measure of Emotioal Itelligece, the MSCEIT (Mayer, Caruso, & Salovey, 2000a, 2000b); ad secod, to assess the reliability ad validity of the Sesatios Test Versio 2. There are, i fact, o uiversal relatios betwee emotios ad autoomic ervous system resposes (Cacioppo, Bertso, Larse, Poehlma, & Ito, 2000) or the physical sesatios that accompay them. Because of this, it is ot possible a priori to specify the right aswers to each emotio-sesatio pair. Emotio researchers i geeral agree that the experiece of emotio is idiosycratic ad that the physical sesatios associated with particular emotios vary from oe perso to aother ad from oe situatio to aother withi the same perso. Therefore, the results of these aalyses will be used to determie both which items validly measure Emotioal Itelligece, as already metioed, ad also how to score each item. As well, because there are i fact o clear ad uambiguous relatioships betwee particular emotios ad physical sesatios, the Sesatios Test should oly be see as a very rough measure of emotio-sesatio kowledge. Method Participats A total of 219 udergraduate studets (150 female, 69 male) participated i this study i retur for course credit. Participats raged i age from 18 to 49 (mea 22.6, SD 6.2). A majority of participats idetified themselves as White (59%), although a relatively large umber of Asias (15%), Blacks (10%) ad Hispaics (9%) also participated. All participats either spoke Eglish as their first laguage (90%) or had bee speakig Eglish for at least 10 years ad felt very comfortable readig ad writig i Eglish. Measures Maximum-Performace Measures Sesatios Test (Barchard, 2003). The Sesatios Test Versio 1 cosists of 7 emotios (scared, agry, jealous, sad, happy, guilty, ad disgusted) ad 32 physical sesatios. These emotios were selected to cover a wide rage of emotios ad to iclude emotios that are commoly icluded i theories of basic emotios (e.g., Ekma & Friese, 1975; Oatley & Johso-Laird, 1987). Sesatios were geerated by brai-stormig possible physical sesatios associated with each of the seve emotios listed, removig duplicates, ad checkig that each sesatio was agreed to be physical rather tha emotioal. For each emotio-sesatio pair, respodets are asked to idicate the frequecy with which the emotio is associated with the sesatio. A 6-poit frequecy scale was used, with 1 idicatig the sesatio is ever associated with that emotio, ad a 6 idicatig it is always associated with that emotio. Mayer-Salovey-Caruso Emotioal Itelligece Test (MSCEIT; Mayer, Caruso, & Salovey, 2000a, 2000b). The MSCEIT is a maximum-performace test of Emotioal Itelligece. It cotais eight subscales that are grouped ito four braches: Perceptio, Facilitatio, Uderstadig, ad Maagemet. The total score o the MSCEIT is calculated by combiig all four subscales. The MSCEIT does ot cotai ay subscales desiged to assess the ability to perceive oe s ow emotios. However, it does cotai two subscales desiged to measure uderstadig of emotios, ad it may be that these scales have the highest correlatios with the Sesatios Test. Geeral Itelligece. Participats were categorized ito high ad low itelligece groups based upo four idicators of Geeral Itelligece: their Verbal ad Math SAT scores, their ACT scores, ad their overall GPA. First, we calculated the average SAT score. Secod, we calculated the z-scores for average SAT scores, for ACT scores, ad for GPA. Third, we calculated the average of these three z-scores for each participat. Fially, participats were categorized ito the two groups based upo a media split: The high itelligece group had Geeral Itelligece scores that were above the media while the low itelligece group had scores that were below the media. It should be oted, however, that the low itelligece group is likely still above average i itelligece compared to the geeral populatio: They were classified as low oly because they were below the media for the group of uiversity studets icluded i this study. Self-Report Measures Style i the Perceptio of Affect Scale (SIPOAS; Beret, 1996). The SIPOAS is a 93-item ipsative measure of persoal prefereces for each of three styles of emotio perceptio. The first subscale, Based o Body, reflects a effortless, itegrated awareess of the fie uaces of body feeligs that precede or accompay the awareess of emotio (Beret, 1986, p. 4) ad Beret argues that it is closely related to Emotioal Itelligece. Emphasis o Evaluatio reflects a style i which great effort is made to uderstad what is happeig to oeself, ofte from the viewpoit of a outside observer, or i terms of imagied ideals or expectatios (Beret, 1986, p. 4). Lookig to Logic iterposes logic betwee feeligs ad respose, to cotrol or avoid potetial discomfort or ambiguity of emotios (Beret, 1986, p. 4). Coceptually, the Based o Body scale is closely associated with the ability to idetify oe s ow emotios. People who report a preferece for usig physical sesatios to uderstad their ow

3 Sesatios Test 3 emotios should kow the relatioships betwee physical sesatios ad emotios better tha people who use exteral stadards or logic to make coclusios about the emotios they are feelig. Toroto Alexithymia Scale 20 (TAS-20; Bagby, Parker, & Taylor, 1994; Bagby, Taylor, & Parker, 1994). The TAS-20 is a 20-item self-report measure of alexithymia. Alexithymia is a cliical coditio associated with a iability to describe emotioal experieces, ad is ofte coceptualized as the opposite of Emotioal Itelligece. The TAS-20 has three subscales: Difficulty Idetifyig Feeligs, Difficulty Describig Feeligs, ad Exterally- Orieted Thikig. Each item is aswered usig a five-poit Likert-type scale. For each subscale, high scores idicate a low level of Emotioal Itelligece. The Difficulty Idetifyig Feeligs scale i particular should have a strog egative relatioship with the Sesatios Test, because it is iteded to measure a iability to idetify oe s ow emotios. Trait Meta-Mood Scale (TMMS; Salovey, Mayer, Goldma, Turvey, & Palfai, 1995). The TMMS assesses people s tedecy to atted to their moods ad emotios, [to] discrimiate clearly amog them, ad [to] regulate them (Salovey et al., 1995, p. 128.) It cotais 30 five-poit Likert-type items fallig o three scales: Attetio, Clarity, ad Repair. The Clarity subscale i particular should measure the same costruct as the Sesatios Test, the ability to correctly idetify oe s ow emotios. Statistical Aalysis Selectig Items for Sesatios Test Versio 2 A variety of statistical criteria were used to select items for iclusio o the secod versio of the Sesatios Test. First, we calculated the correlatios betwee scores o the 224 items o the Sesatios Test with two criterio measures: scores o the MSCEIT Uderstadig subscale ad total scores o the MSCEIT. Items with sigificat correlatios with oe or both of these scales were tetatively idetified as measurig Emotioal Itelligece. These aalyses were the repeated for the high ad low Geeral Itelligece groups, to esure that the items that were selected worked well with a wide rage of itelligece levels. Oe-way ANOVA s ad scatterplots were also examied, to explore possible o-liear relatioships or correlatios that might be uduly iflueced by oe or two outliers. The Sesatios Test Versio 2 was the compiled usig the emotio-sesatio pairs that appeared from these five aalyses to have the most compellig evidece that they measured Emotioal Itelligece. Creatig the Scorig Key for Sesatios Test Versio 2 The scorig key for the Sesatios Test Versio 2 was created by examiig the directio of the correlatios betwee the items ad total scores o the MSCEIT. If the correlatio was positive, the item was positively-keyed. If the correlatio was egative, the item was egatively-keyed. If there was o relatioship, the item was ot scored. Validatig Sesatios Test Versio 2 To assess the reliability ad validity of the Sesatios Test Versio 2, two statistical aalyses were coducted. First, the iteral cosistecy of Versio 2 was examied ad a item aalysis was coducted to determie if ay items were detractig from iteral cosistecy. Secod, we calculated the correlatios betwee total scores o the Sesatios Test Versio 2 ad 5 maximum-performace ad 9 self-report measures of Emotioal Itelligece. The 5 maximum-performace measures were the four scale scores of the MSCEIT ad the total scores from the MSCEIT. The 9 self-report measures were the subscales of the SIPOAS, TMMS, ad TAS-20 scales. Procedures Participats completed the above-metioed measures i two oe-hour group testig sessios scheduled oe week apart. All measures were admiistered o the computer, uder the supervisio of a traied research assistat. Results Developmet of Versio 2 To determie which items to iclude o the Sesatios Test Versio 2, each of the 224 emotio-sesatio pairs were examied based o the aalyses described above. For 20 of these pairs, there was strog evidece that scores o the emotio-sesatio pair were associated with Emotioal Itelligece, as measured by the MSCEIT Uderstadig subscale ad MSCEIT Total Scores (see Table 1). However, the emotio disgusted had o compellig relatioships with ay of the sesatios, ad guilty oly had a sigle compellig relatioship. Therefore, these two emotios were omitted from Versio 2 of the Sesatios Test. The remaiig five emotios were each paired with the 11 idetified sesatios to create a 55-item test, where 19 of the items would be scored. See the Appedix for Versio 2 of the Sesatios Test. Iteral Cosistecy Item Aalysis of Versio 2 The iteral cosistecy of the 19-item Sesatios Test Versio 2 was calculated as.66. This level of iteral cosistecy is lower tha would be ideally desired, but adequate for research purposes. Corrected Item-Total Correlatios ad Alpha-if-Item-Deleted were the examied to determie which items were cotributig to this low iteral cosistecy. The followig emotio-sesatio pairs decreased iteral cosistecy by a small amout: Scared Tese Muscles, Agry Weak, ad Happy Lighthearted.

4 Sesatios Test 4 Coverget Validity of Versio 2 The coverget validity of the 19-item Sesatios Test Versio 2 was assessed by calculatig the correlatios of total scores o the Sesatios Test with 5 maximum-performace ad 9 self-report measures of Emotioal Itelligece (see Table 3). The Sesatios Test Versio 2 has moderate positive correlatios with each of the five MSCEIT measures. These correlatios should be cosidered over-estimates of the true correlatios betwee Versio 2 ad the MSCEIT, because the MSCEIT was used durig the selectio of items for the scale. O the other had, it is ecouragig that all of the brach scores have moderate positive correlatios with Versio 2, eve though oly the MSCEIT Uderstadig ad MSCEIT Total scores were used durig item selectio. Most of the self-report measures of Emotioal Itelligece did ot have sigificat correlatios with the Sesatios Test. Whe we desiged this study, we hoped that the Sesatios Test would be positively correlated with self-report measures of Emotioal Itelligece, particularly those subscales that were desiged to measure the ability to idetify oe s ow emotios. However, i retrospect, the lack of correlatio is ot very surprisig give that previous research (Barchard & Hakstia, 2003) has show that self-report ad maximum-performace measures of Emotioal Itelligece have very little relatioship with each other. These results could therefore be iterpreted as testimoy to the importace of creatig a maximum-performace measure of this skill, rather tha as evidece that the Sesatios Test lacks coverget validity. However, the Sesatios Test did have statistically sigificat correlatios with two self-report measures of Emotioal Itelligece, both subscales of the SIPOAS. The SIPOAS Based o Body scale was desiged to measure a cogitive style that is the core of Emotioal Itelligece, ad Emphasis o Evaluatio ad Lookig to Logic were set up as alterative cogitive styles. These alterative styles should ot be associated with high Emotioal Itelligece. Istead, a o-sigificat or egative correlatio with these two subscales would have bee expected. The SIPOAS Lookig to Logic subscale did ideed have a small egative correlatio with the Sesatios Test (r = -.172, p <.01). However, the Based o Body subscale did ot have the sigificat positive correlatio expected, ad the correlatio with the Emphasis o Evaluatio subscale was positive (r =.183, p <.01) istead of egative, which was very surprisig. Not oly did the Sesatios Test fail to correlate with the Sesatios Test i the maer expected, but it also did ot correlate i the same way that other self-report measures did: Other self-report measures of Emotioal Itelligece did ot have sigificat correlatios with the Sesatios Test at all. We therefore coclude that the uexpected correlatio betwee the Sesatios Test ad the SIPOAS Emphasis o Evaluatio subscale may poit to a weakess i the SIPOAS rather tha the Sesatios Test. Coclusios The purpose of this research was to create a measure of Emotioal Itelligece by selectig emotio-sesatio pairs that validly assess kowledge of the relatio betwee emotios ad physical sesatios. Of the 224 emotiosesatio pairs that were examied, the vast majority showed o particular relatioship with Emotioal Itelligece: It was ot the case that Emotioally Itelliget people kew that these emotios were associated with these physical sesatios or that they were ot associated. Istead, for may emotios, everyoe seemed to agree that there was or was ot a relatioship betwee a particular emotio ad a particular physical sesatio. Most of these items were, therefore, too easy for the populatio used i this study, college studets, eve whe focusig o those college studets i our sample who had lower levels of Geeral Itelligece. However, we were able to idetify some emotio-sesatio pairs that appear to measure Emotioal Itelligece. First, we idetified emotio-sesatio pairs that had moderate relatioships with Emotioal Itelligece. Next, we elimiated items that appeared to have o-liear relatioships with Emotioal Itelligece, ad those where the relatioships appeared depedet upo a small umber of outliers. Whe these problematic items were elimiated, we were left with a hadful of emotio-sesatio pairs that provide compellig evidece that they measure Emotioal Itelligece. The Sesatios Test Versio 2 was created usig 5 emotios ad 11 physical sesatios, for a total of 55 items of which 19 are scored. It has acceptable iteral cosistecy ad some evidece of coverget validity. The ability to idetify oe s ow emotios is arguably the most cetral skill i the area of Emotioal Itelligece. The Sesatios Test represets a ovel approach to assessig this skill. However, additioal research is eeded o this ew measure before its use i applied settigs ca be justified. The relatioships of Versio 2 with the MSCEIT eed to be cross-validated i a ew dataset, ad the relatioships with additioal measures of Emotioal Itelligece, related cocepts, ad relevat criterio-measures eed to be examied.

5 Sesatios Test 5 Refereces Bagby, R. M., Parker, J. D. A., & Taylor, G. J. (1994). The twety-item Toroto Alexithymia Scale. I. Item selectio ad cross-validatio of the factor structure. Joural of Psychosomatic Research, 38, Bagby, R. M., Taylor, G. J., & Parker, J. D. A. (1994). The twety-item Toroto Alexithymia Scale. II. Coverget, discrimiat, ad cocurret validity. Joural of Psychosomatic Research, 38, Barchard, K.A. (2003). Sesatios Test Versio 1. Upublished psychological test. Available from Kim Barchard, Departmet of Psychology, Uiversity of Nevada, Las Vegas, 4505 S. Marylad Parkway, P.O. Box , Las Vegas, NV, , USA, barchard@ulv.evada.edu Barchard, K.A. & Hakstia, A.R. (i press). The ature ad measuremet of emotioal itelligece abilities: Basic dimesios ad their relatioships with other cogitive-ability ad persoality variables. Educatioal ad Psychological Measuremet. Bar-O, R. (1997). EQ-i: BarO emotioal quotiet ivetory, techical maual. Toroto, ON: Multi-Health Systems. Beret, M. (1996, August). Emotioal itelligece: Compoets ad correlates. Paper preseted at the 104 th Covetio of the America Psychological Associatio, Toroto, Caada. Cacioppo, J.T., Bertso, G.G., Larse, J.T., Poehlma, K.M. & Ito, T.A. (2000). The psychophysiology of emotio. I M. Lewis ad J. M. Havilad-Joes (Eds.), Hadbook of emotios (2 d ed, pp ). New York: Guilford Press. Ekma, P. & Friese, W.V. (1975). Umaskig the face: A guide to recogizig emotios from facial clues. Oxford, Eglad: Pretice-Hall. Mayer, J.D., Caruso, D.R., & Salovey, P. (2000a). Emotioal itelligece meets traditioal stadards for a itelligece. Itelligece, 27, Mayer, J.D., Caruso, D.R., & Salovey, P. (2000b). Istructio maual for the MSCEIT, Mayer, Salovey-Caruso emotioal itelligece test, research versio 1.1 Toroto, ON: Multi-health Systems. Oatley, K., & Johso-Laird, P. N. (1987). Towards a cogitive theory of emotios. Cogitio ad Emotio, 1, Salovey, P., Mayer, J. D., Goldma, S. L., Turvey, C., & Palfai, T. P. (1995). Emotioal attetio, clarity, ad repair: Explorig emotioal itelligece usig the trait meta-mood scale. I D. M. Weger & J. W. Peebaker (Eds.), Emotio, disclosure, ad health (pp ). Washigto, DC: America Psychological Associatio. Schutte, N., Malouff, J. M., Hall, L. E., Haggerty, D. J., Cooper, J. T., Golde, C. J., & Dorheim, L. (1998). Developmet ad validatio of a measure of emotioal itelligece. Persoality ad Idividual Differeces, 25, Tett, R., Wag, A., & Fox, K. (2003). Emotioal Itelligece as a Multidimesioal Persoality Costruct: Developmet ad Validatio of a Self-Report Measure. Mauscript i preparatio. Available from Robert Tett, Departmet of Psychology, 308 Lorto Hall, Uiversity of Tulsa, 600 S. College, Tulsa, OK

6 Sesatios Test 6 Table 1 Positively Associated ad Negatively Associated Emotio-Sesatio Pairs Scared Disgusted Guilty Sad Agry Jealous Happy Tese muscles p Refreshed Heart beats faster p Jumpy p Sexually aroused Weak Cold Lighthearted p Gaggig Strog Eergetic Note. p idicates a compellig positive relatioship betwee this emotio-sesatio pair ad Emotioal Itelligece, while idicates a compellig egative relatioship. Table 2 Item Aalysis of the Sesatios Test Versio 2 Emotio-Sesatio Pair Corrected Item Total Correlatio Alpha if Item Deleted Scared Tese Muscles Scared Heart Beats Faster Scared Jumpy Scared Sexually Aroused Scared Refreshed Sad Refreshed Sad Sexually Aroused Agry Weak Agry Cold Agry Lighthearted Agry Refreshed Agry Sexually Aroused Jealous Sexually Aroused Jealous Eergetic Jealous Gaggig Jealous Strog Jealous Refreshed Happy Jumpy Happy Lighthearted

7 Sesatios Test 7 Table 3 Correlatios of Sesatios Test Versio 2 with Maximum-Performace ad Self-Report Measures of Emotioal Itelligece Correlatios Maximum-Performace Measures MSCEIT Perceptio.411** MSCEIT Facilitatio.450** MSCEIT Uderstadig.437** MSCEIT Maagig.404** Total MSCEIT.529** Self-Report Measures SIPOAS Lookig to Logic -.172* SIPOAS Based o Body SIPOAS Emphasis o Evaluatio.183** TAS20 Difficulty Idetifyig Feeligs TAS20 Difficulty Describig Feeligs.024 TAS20 Exterally Orieted Thikig TMMS Repair TMMS Attetio.033 TMMS Clarity * p <.05. ** p <.01.

8 Sesatios Test 8 Appedix Sesatios Test Versio 2 Strog emotios are associated with distictive physical sesatios. This is a test of your kowledge of the relatio of physical sesatios to emotios. This test focuses o four emotios feelig scared, agry, jealous, or happy ad 11 differet physical sesatios. For each combiatio, please idicate how ofte these physical sesatios are associated with each of the followig emotios. Be sure to aswer every questio. If you leave a questio blak, it is automatically wrog. Use the followig scale: Never Rarely Sometimes Ofte Usually Always Scared Agry Jealous Sad Happy Tese muscles Refreshed Heart beats faster Jumpy Sexually aroused Weak Cold Lighthearted Gaggig Strog Eergetic