Bimodal Score Distributions and the MBTI: Fact or Artifact?

Transcription

1 Paper presented at the 2001 (Aprl Annual Conference of the Socety for Industral and Organzatonal Psychology, San Dego. Bmodal Score Dstrbutons and the MBTI: Fact or Artfact? Tammy L. Bess and Robert J. Harvey Vrgna Polytechnc Insttute and State Unversty We examned Myers-Brggs Type Indcator (MBTI score dstrbutons computed usng temresponse theory (IRT to assess the generalzablty of earler bmodalty reports whch have been cted n support of the type versus trat vew of personalty. Usng the BILOG IRT program to score a sample of approxmately 12,000 ndvduals who partcpated n leadershp development programs, θ score dstrbutons for the four dmensons of the MBTI computed usng 10 (the BILOG default versus 50 uadrature ponts were compared. Results ndcated that past reports of bmodalty were artfacts caused by BILOG s default use of a small number of uadrature ponts; when larger numbers of ponts were used, score dstrbutons became strongly center-weghted. Although our fndngs are not supportve of the type -based hypothess, the extremely hgh correlatons between θ scores (rs >.996 suggest that no practcal dfferences would be expected as a functon of the number-of-uadrature-ponts decson. The Myers-Brggs Type Indcator (MBTI remans hghly popular n appled organzatonal settngs (e.g., Myers, McCaulley, Quen, & Hammer, 1998, despte the fact that ts Jungan type -based vew of personalty s conceptually ute dfferent from the trat or contnuous dmensonal vew of personalty popularzed by other nstruments, and n the Fve Factor Model (FFM vew of the structure of personalty (e.g., Goldberg, 1992; McCrae & Costa, 1987, Although the MBTI has always been scored to produce contnuous preference scores for each of ts four dmensons and ndeed, n ts most recent revson the MBTI adopted a contnuous dmensonal scorng system based on the 3- parameter logstc tem response theory (IRT model ts developers contnue to emphasze the necessty of usng dchotomous types when mang assessment decsons, and not the contnuous scores on the dmensons themselves that form the bass for the dchotomzed types. The MBTI was founded on Jung s (1921/1979 theory of psychologcal type, and focuses on the four dchotomes that were mplct or explct n Jung s theory. Jung orgnally proposed that dfferences n behavor can be attrbuted to whether people are ntroverted or extraverted (the E-I dmenson n the MBTI, whch parallels the Extraverson scale of the FFM; ths dstncton was based on whether a person s energes were prmarly drected toward the nner world of thought and experence (ntroverson, or orented towards other people and stuatons (extraverson. Later, Jung extended hs theory to nclude two addtonal dchotomes: the sensaton (or sensng-ntuton (S-N, whch parallels the FFM s Openness dstncton focused on the mental functonng of percevng, and the thnng-feelng dchotomy (T-F, analogous to the FFM s Agreeableness focused on the judgng functons. Durng the development of the MBTI (e.g., Myers & Brggs, 1962, the judgng-percevng (J-P, analogous to the FFM s Conscentousness dchotomy whch was largely mplct n Jung s theory was added, based upon Myers unpublshed typologcal wor (e.g., see Myers et al, Despte the strong theoretcal foundaton and the fact that the MBTI has been wdely used, debate s ongong wth respect to the ueston of whether dscrete personalty types actually exst (e.g., Bloc & Ozer, 1982; Mendelsohn, Wess, & Femer, 1982; Mller & Thayer, 1989; Strcer & Ross, Unfortunately, at least wth respect to the tradtonal preference-score method of scorng the MBTI, research has consstently shown that the bmodal score dstrbutons mpled by the type vew of personalty are not typcally present n large, unselected populatons of examnees. Although the absence of bmodal score dstrbutons does not necessarly prove that the type -based approach s ncorrect, f such dstrbutons were to be found, ths fact would defntely be cted as support for the MBTI s underlyng typebased approach. Indeed, Myers and McCaulley (1985, pg. 157 agreed on the smple attractveness of beng able to fnd bmodal dstrbutons. The advent of an IRT-based scorng procedure for the MBTI (Harvey & Murry, 1994 was sgnfcant for a varety of reasons, wth one unntended effect beng that nstead of producng the strongly center-weghted score dstrbutons that had long been seen for the preference-score method, the θ (theta scores produced by IRT nstead exhbted a defnte bmodal nature, wth a relatvely low densty of subjects scorng n the mddle of the dstrbuton (.e., at the type dchotomzaton pont. Ths fact has been cted n support of the dchotomous type-based feedbac model used by the MBTI (e.g., Myers et al., However, although Harvey and Murry (1994 dd report that the IRT scorng system produced bmodal dstrbutons, they also noted that the dstrbutons were not especally sharply bmodal (p. 126, and stopped short of drawng the concluson that ther fndngs provded defntve evdence regardng the type versus trat controversy, callng nstead for addtonal research n new samples. Our study responded to thes need for a further nvestgaton of the bmodalty ssue wth respect to IRT scorng of the MBTI. Frst, although the Harvey and Murry

2 Tammy L. Bess & Robert J. Harvey (1994 sample was szable, t was not especally dverse, wth college students representng the majorty of the approxmately 1,600 partcpants. Although college students clearly are people, and they arguably represent acceptable subjects for personalty research, one can nevertheless ueston the degree to whch results especally, unprecedented results obtaned n a student-domnated sample would generalze to other populatons (n partcular, samples composed of ndvduals who complete the MBTI n a wor-related, rather than a research-based context. Second, t s possble that the bmodalty reported by Harvey and Murry (1994 mght have been an unntended artfact of the specfc IRT scorng methodology used n that study. That s, Harvey and Murry (1994 used the default scorng parameters for EAP scorng wth a normal pror dstrbuton provded by the BILOG program (Mslevy & Boc, 1990 when estmatng the θ scores. Although the BILOG program defaults would presumably be reasonable, one parameter n partcular the number of uadrature ponts used durng the scorng process may have had the potental to nfluence the shape of the dstrbutons of θ scores. That s, n the context of bnary IRT models, tem-response probabltes are expressed as a jont functon of the IRT scorng parameters a, b, and c for each tem, at a gven value of θ (D s a scalng constant, typcally set to 1.702: P ( θ c + ( c Da ( θ b e 1 Da ( b [1] = θ 1+ e To estmate a θ score for each ndvdual, a maxmum lelhood functon can be derved from the expected tem-endorsement probablty specfed n [1] as follows (n = number of tems, u = tem response, and Q = 1-P: l ( u θ lnl = n = 1 P n ( u θ = [ ( u ln P + [( 1 u ] ln( 1 P ] = 1 u Q 1 u (In practce, the log of the lelhood value s usually maxmzed nstead of drectly attemptng to maxmze the raw lelhood; here, u denotes the 1/0 response for each tem (1 f the response was n the I, N, F, or P drecton, and 0 otherwse. In BILOG, the θ score that maxmzes the lelhood (or log-lelhood functon s computed usng a uadrature-pont method: θ = PSD = 1 ( θ x = 1 L = L( x w( x ( x w( x = 1 2 ( x θ L( x w( x L( x w( x = 1 1/ 2 [2] [3] 2 In ths method, the lelhood functon s evaluated at only a small subset of eually spaced ponts x along the θ scale (.e., the uadrature ponts, each wth an assocated uadrature weght (w, and the θ estmate s computed smply by weghtng and summng these values across the uadrature ponts (the PSD ndexes the standard error of the θ estmate. Although the choce of the number of uadrature ponts mght not necessarly be expected to produce apprecably dfferent θ estmates, upon comparng the locatons of the default uadrature ponts used by BILOG for the MBTI (.e., at θ = -4.0, -3.1, -2.2, -1.3, -0.4, 0.4, 1.3, 2.2., 3.1, and 4.0 aganst the bmodal θ dstrbutons reported by Harvey and Murry (1994, we were struc by the fact that the areas of hghest densty corresponded closely to the locatons of the two mddle uadrature ponts. Hence, the second objectve of our study was to compare the earler scorng method usng 10 uadrature ponts aganst the θ scores produced usng a much larger number of uadrature ponts (.e., 50. By usng a larger number of ponts, ths second method would more closely approxmate the results that would be obtaned n a bruteforce approach of evaluatng the lelhood functon, and allow for a dfferent shaped dstrbuton to emerge f t were emprcally so dsposed (.e., by locatng a number of ponts throughout the mddle range of the θ scale, rather than havng only two ponts spaced nearly a standard devaton apart, wth no pont at the mddle of the θ scale. Partcpants Method The MBTI responses used n the present study were obtaned from a large, non-proft leadershp tranng and development organzaton. After elmnatng mssng data (for each scale, profles were dscarded f they contaned any mssng responses for the tems n that scale, the fnal datasets were as follows: the EI scale contaned 11,789 subjects, TF scale contaned 12,338 subjects, SN scale contaned 12,195 subjects, and JP scale contaned 12,316 subjects. All subjects were managers n organzatons tang the MBTI as part of a leadershp development program. Procedure All data were analyzed usng BILOG verson 3.07 (the verson used n Harvey & Murry, 1994 as well as the newer BILOG-MG verson 1.1c. In order to examne the effect on θ score dstrbutons due to the number of uadrature ponts, scorng analyses were run usng the default number of ponts for scales of the length seen n the MBTI (10, as well as usng the maxmum-possble number of ponts (50. After calbratng the tems and scorng them usng EAP and the normal pror dstrbuton (a unform pror dstrbuton,.e., plan unweghted maxmum-lelhood estmaton, was also evaluated, and hghly smlar results were produced; to conserve space, only the EAP results are reported, θ score freuency dstrbutons were examned for each of the four

3 scales, and correlatons were computed between the θ estmates for each scale. Results Wth regard to the ueston of whether the number-ofuadrature-ponts decson may have nfluenced the Harvey and Murry (1994 report of bmodal score dstrbutons for the MBTI, the results n Fgures 1, 3, 5, and 7 depct the unvarate freuency dstrbutons produced usng the BILOG default of 10 uadrature ponts, whereas those n Fgures 2, 4, 6, and 8 depct the dstrbutons produced usng the maxmum of 50 uadrature ponts. As n the Harvey and Murry study, the θ dstrbutons for the default uadrature ponts produced a clearly bmodal shape. Addtonally, as n the earler study, the locatons of the prmary modes are located ute close to the θ = 0.44 and 0.44 uadrature ponts, and less-pronounced secondary modes can be observed at approxmately θ = 1.3 and 1.3 (.e., the locatons of the adjacent uadrature ponts. However, comparsons of each scale s dstrbuton computed usng a large number of uadrature ponts (Fgures 2, 4, 6, and 8 aganst those produced usng the default mae t ute clear that the decson regardng the number and locaton of the uadrature ponts exerts a very strong effect on the subseuent shapes of the MBTI θ score dstrbutons. As was found n earler research usng the preference-score based MBTI scales (e.g., Harvey & Murry, 1994; Strcer & Ross, 1964, heavly center-weghted, non-bmodal dstrbutons result when a hgher degree of granularty s used when evaluatng the lelhood functon. Wth regard to the practcal sgnfcance of the uadrature ponts choce, correlatons between the 10- versus 50- uadrature pont θ estmates for the E-I, S-N, T-F, and J-P scales were r =.99808,.99793,.99635, and.99794, respectvely, ndcatng that although the uadrature pont decson clearly exerts a powerful effect on dstrbuton shape, the bottom-lne mpact of t on the θ scores themselves s prmarly one of selectvely stretchng or compressng the θ metrc, wthout producng any meanngful change n the orderng of ndvduals on the θ scale. That s, as Fgure 9 llustrates for the E-I scale, the θ scores produced usng a hgher number of uadrature ponts are essentally a nonlnear transformaton of the θ scores produced usng the smaller number of ponts. Smlar results were observed for the remanng scales. Dscusson Our fndngs ndcate that the enthusasm seen among advocates of the MBTI based on the bmodal score dstrbutons reported by Harvey and Murry (1994 needs to be sgnfcantly tempered n lght of the fact that across all four dmensons, the results from the present study ndcate that the earler reports of bmodalty were essentally artfacts caused by the partcular number (and locaton of uadrature ponts used by default n BILOG. Although we do not conclude that the absence of bmodalty necessarly proves that the MBTI developers theory-based assumpton of categorcal types of personalty s nvald, the absence of emprcal bmodalty n 3 MBTI Bmodalty IRT-based MBTI scores does ndeed remove a potentally powerful lne of evdence that was prevously avalable to type advocates to cte n defense of ther poston. Fortunately, because the man effect of uadrature-pont choce appears to be a relatvely modest, and selectve, shrnng-stretchng of the θ scale around the locaton of each pont, an overwhelmngly strong correspondence exsts between the θ scores estmated usng dfferent numbers of uadrature ponts. Thus, as a practcal matter, we are at a loss to envson a stuaton n whch t would mae much of a practcal dfference whch method were used, gven that the θ score estmates correlate n excess of r =.996 across all four MBTI scales. References Bloc, J., & Ozer, D. J. (1982. Two type of psychologsts: Remars on the Mendelsohn, Wess, and Femer contrbuton. Journal of Personalty and Socal Psychology, 42, Brggs, K. C. & Myers, I. B. (1976. Myers-Brggs Type Indcator: For F. Palto Alto, CA: Consultng Psychologsts Press. Goldberg, L. R. (1992. The development of marers for the Bg Fve factor structure. Psychologcal Assessment, 4, Harvey, R. J., & Murry, W. D. (1994. Scorng the Myers- Brggs type ndcator: Emprcal comparson of preference score versus latent-trat methods. Journal of Personalty Assessment, 62, Mendelsohn, G. A., Wess, D. S., & Femer, N. R. (1982. Conceptual and emprcal analyss of the typologcal mplcatons of patterns of socalzaton and femnnty. Journal of Personalty and Socal Psychology, 42, McCrae, R. R., & Costa, P. T. (1987. Valdaton of the fvefactor model of personalty across nstruments and observers. Journal of Personalty and Socal Psychology, 52, McCrae, R. R., & Costa, P. T. (1989. Renterpretng the Myers-Brggs Type Indcator from the perspectve of the fve-factor model of personalty. Journal of Personalty, 57, Mller, M. L., & Thayer, J. R. (1989. On the exstence of dscrete classes n personalty: Is self-montorng the correct jont to carve? Journal of Personalty and Socal Psychology, 57, Myers, I. B., & McCaulley, M. H. (1985. Manual: A gude to the development and use of the Myers-Brggs type ndcator. Palto Alto, CA: Consultng Psychologsts Press. Myers, I. B., McCaulley, M. H., Quen, N. L., & Hammer, A. L. (1998. Manual: A gude to the development and use of the Myers- Brggs type ndcator. Palo Alto, CA: Consultng Psychologsts Press. Spps, G. J., Alexander, R. A., & Fredt, L. (1985. Item Analyss of the Myers-Brggs type ndcator. Educatonal and Psychologcal Measurement, 45, Strcer, L. J., & Ross, J. (1964. Some correlates of a Jungan personalty nventory. Psychologcal Reports, 14, Tzeng, O. C. S., Ware, R., & Bharadwaj, N. (1991. Comparson between contnuous bpolar and unpolar ratngs of the Myers-Brggs type ndcator. Educatonal and Psychologcal Measurement, 51, Tzeng, O. C. S., Ware, R., & Chen, J. M. (1989. Measurement and utlty of contnuous unpolar ratngs for the Myers-Brggs type ndcator. Journal of Personalty Assessment, 53,

4 Fgure 1 Freuency theta score estmates usng default uadrature ponts for the EI scale. Fgure 2. Freuency theta-score estmates usng ffty uadrature ponts for the EI scale. Fgure 3. Freuency theta score estmates usng default uadrature ponts for the TF scale.

5 Fgure 4. Freuency theta-score estmates usng ffty uadrature ponts for the TF scale. Fgure 5. Freuency theta score estmates usng default uadrature ponts for the SN scale. Fgure 6. Freuency theta-score estmates usng ffty uadrature ponts for the SN scale. 5

6 Tammy L. Bess & Robert J. Harvey Fgure 7. Freuency theta score estmates usng default uadrature ponts for the JP scale. Fgure 8. Freuency theta-score estmates usng ffty uadrature ponts for the JP scale. Fgure 9. Scatterplot of the EI scale results computed usng 10 versus 50 uadrature ponts. 6