Construct Validity of the MBTI in Management Development: A Test of Two Interpretations. Robert B. Kaiser & S. Bartholomew Craig

Transcription

1 Construct Validity of the MBTI in Management Development: A Test of Two Interpretations Robert B. Kaiser & S. Bartholomew Craig

2 Myers-Briggs Type Indicator (MBTI) Derived from an explicit theory Scales purport to measure types not continuous traits The take home is a gestalt person is classified as 1 of 16 categorically unique personality types

3 The whole of what test scores mean Do test scores relate to other variables in the structural form stated or implied by the theory from which the measure was derived? Judgment made on a body of evidence bearing on the question, What inferences about scores can be justified? Does the MBTI measure types or traits?

4 Behavior = f (EvsI x SvsN x TvsF x JvsP) 16 unique personality types (2 x 2 x 2 x 2) 4 dichotomous type preferences E-I S-N T-F J-P attitude for directing energy (outward/inward) perceiving function (acquiring information) judging function (for making decisions) attitude for dealing with external world (Dom./Aux.)

5 Forced-choice items matched pairs for each bipolar preference Pref A Pref B O I like parties O I like solitude Scoring Sum endorsements for preferences A & B separately Highest score (A vs. B) is type preference Continuous score -- linear transformation of (A-B)

6 4 dichotomous preference scales Opposing poles are qualitatively unique Either/or, not less to more 16 personality types Behavior = f (EvsI x SvsN x TvsF x JvsP) Synergy a type is more than the sum of its parts

7 Univarate, bi-polar scale discontinuity at cut-point when related to a criterion variable A B C Criterion score Criterion score Criterion score Cut-point Preference A Preference B Cut-point Preference A Preference B Cut-point Preference A Preference B Multivariate typology Component variables interact when related to criteria Requires factorial design to test (one-way won t do)

8 Viewed through lens of FFM as continuous traits EI E - E factor; - dominance, - sociability SN O/I + O/I factor; + creativity, + independence TF A + A factor, - Open to ideas, + Open to feelings JP C/O - C, + O factors, - prudence, - methodical, + spontaneity

9 Which view of the MBTI is most supported by data? Types or Traits Predict 360 ratings using statistical models that 1) assume the MBTI measures types 2) assume the MBTI measures traits Let the preponderance of evidence be the judge of what MBTI scores mean (construct validity)

10 Participants of a leadership program, ,084 focal managers 64% male 25% middle, 45% GMs, 25% Sr. executives MBTI form F 360 ratings Benchmarks 8,720 superiors 27,098 peers 26,982 subordinates Due to large N, used p =.001 and minimum effect size R 2 >.01

11 EFA, SME judgment, CFA model fit: CFI =.90, NFI =.90, RMSEA =.05 Relationship Orientation (13 items, α=.93) Empowering, Participative Orientation (11 items, α=.89) Work-Life Balance (4 items, α=.82) Decisive, Action-orientation (12 items, α=.89) Mastery Learning Orientation (5 items, α=.85) Lack of Follow Through (4 items, α=.79) Overall Effectiveness (16 items, α=.93)

12 Discontinuities different slopes A Criterion score Preference A Preference B MBTI Continuous score Compare β s from regression equations for each opposing type 2 (linear, curvilinear) x 4 (scales) x 7 (performance ratings) x 4 (rating groups) = 224 tests 14 indications of differential slope 3 where rate of change occurred near putative midpoint effects were significant p <.001, effect sizes were trivial

13 4 Work-Life Balance (subordinate rating) Observed Linear Quadratic 3 Cubic SN continuous score

14 5 Work-Life Balance (subordinate rating) 4 3 Observed 2 Linear Quadratic 1 Cubic SN continuous score

15 Discontinuities different predicted values at cut-point B Criterion score Preference A Preference B MBTI Continuous score Compute predicted criterion value for X = 100 from regressions in opposing types, form 99.9% Confidence Interval 2 (linear, curvilinear) x 4 (scales) x 7 (performance ratings) x 4 (rating groups) = 224 tests 7 instances of significant differences in predicted criterion value 0 instances that could account for greater % variance in type analysis over trait analysis

16 Interactions between type variables 2 x 2 x 2 x 2 MANOVA predicting 28 criteria 2 way interactions: 16 3 way interactions: 8 4 way interactions: 1 Total interactions tested: 28 scales x 25 terms = 700 effects Number significant: 1 EI x TF predicting Work-Life bal. (subs) EFs rated highest, ITs lowest, IF & EF the same Not predicted by type theory, trivial effect (.008)

17 The bottom line: Comparison of variance accounted for across 28 criteria Interactive Type Model Additive Trait model Multivariate effects partial eta 2 R 2 EI SN TF JP EI x TF.008

18 Substantive findings in terms of FFM Scale EI -E Aspect of Managerial performance related to Robust predictor of self-ratings - Relationship Orientations & - Decisive Action-orientation SN O/I + Mastery learning orientation TF A + Relationship orientation, + Participative, + W-L balance JP -C/O + Lack of follow through Note: Effect sizes ranged from.00 to.048.

19 MBTI traits x 360 ratings correlations Self Superior EI SN TF JP EI SN TF JP Effectiveness Relationship Orientation Empowering, Participative Orientation Work-Life Balance Decisive, Action-Orientation Master Learning Orientation Lack of Follow Through Peers Subordinates EI SN TF JP EI SN TF JP Effectiveness Relationship Orientation Empowering, Participative Orientation Work-Life Balance Decisive, Action-Orientation Master Learning Orientation Lack of Follow Through

20 The preponderance of empirical evidence suggests, that in the context of predicting managerial performance: The MBTI is not a valid measure of the 16 group typology construct The 4 MBTI scales are not valid measures of dichotomous type constructs The four MBTI scales are best considered as measures of 4 continuous trait constructs that can be interpreted in terms of the FFM