Standardization of the Dutch MCMI-III: Specific problems associated with the use of base rates

Transcription

1 Standardization of the Dutch MCMI-III: Specific problems associated with the use of base rates Gina Rossi & Hedwig Sloore Corresponding author: Rossi Gina Commission, July 6-8, 2006 Faculty of Psychology & Educational Sciences Department of Personality and Social Psychology

2 MCMI-III (Millon et al., 2006) Millon Clinical Multiaxial Inventory - Self-report instrument (175 items) - 27 scales 3 modifier scales (like validity scales) e.g. Y desirability 14 personality disorder scales (DSM axis II) e.g. 6A antisocial personality disorder 10 clinical syndrome scales (DSM axis I) e.g. CC major depression

3 Translation of U.S. questionnaires - Main problem: question of equivalence - Equivalence can be achieved using specific methods during translation and evaluation (Butcher, 1996)

4 Europe versus United States - U.S.: Domain of psychopathology strongly influenced by DSM (APA) - Europe: No problems in using DSM classification - Western culture : psychological differences nonexistent or of no importance => Use of U.S. questionnaires

5 Standardization Dutch MCMI-III - Translation-back-translation (method Butcher, 1996) - Details Dutch translation (Derksen & Sloore, 2005) - Specific problem: transformation of raw scores is based on the base rate of the different forms of pathology -> prevalence often differs across countries!

6 Millon: use of base rates Norm referencing implies normal distributions and comparable frequencies BUT MCMI-III scales have skewed distributions: e.g. very few patients with high scores on the schizotypal disorder scale Frequencies differ: e.g. a lot more patients with depression than with delusional disorder

7 U.S. Base rates (Millon et al., 1997) - Cut-off score for each scale anchored to the real prevalence ratio - Some anchor points: BR 60 = median BR 85 = disorder level If e.g. 70% of the patients has a depression, the raw score corresponding to the 70th percentile will be transformed into BR 85

8 Belgian Base rates - Prevalence BR are problematic: patients with a particular disorder do not always obtain the highest score on the scale (e.g. limited insight, denial) - Therefore: use of Receiver Operating characteristic Curve (ROC) analysis

9 ROC analysis - Recommended in recent clinical literature (e.g. Hsu, 2002; Nurnberg et al., 2000; Swets et al. 2000, ) - The curve represents the ratio of sensitivity and specificity

10 Good diagnostic instrument Sensitivity and specificity above.70 (West & Finch, 1997) When optimization is not possible: priority to sensitivity -> MCMI-III in Europe/Belgium used for SCREENING - Sensitivity = probability of a test being positive if a disorder is present (true positive cases) - Specificity = probability of a test being negative in the absence of any disorder (true negative cases)

11 Sample Final data set with only valid MCMI-III s (n=524) males and 269 females - majority of clinical setting (n=438 or 83.6%) - smaller group from forensic setting (n=86 or 16.4%)

12 Example males: ROC curve scale 3 (dependent PD) - disorder level

13 Coordinates of the ROC curve If * Sensitivity Specificity * The smallest cut-off value is the minimal observed test value minus 1, the greatest cut-off value is the maximal observed test value plus 1. Every other cut-off value is the mean of two subsequent observed test values

14 Titel van de Slide Matching ROC base rates to raw scores Raw score Note: Anchor points appear in red Frequency Cumulative % BR anchor points and linear interpolation Round numbers final BR

15 General results (Rossi & Sloore, 2005) Sensitivity Mean.71 Range.53 (8B self-defeating PD).84 (6B sadistic PD) Specificity Mean.56 Range.41 (N bipolar) -.82 (SS thought disorder)

16 Conclusion : Disadvantages 1. Need criterion group -> reliability clinical evaluations important! 2. Good screening instrument, but a diagnosis can be made only on the basis of supplementary information

17 Conclusion: Advantages 1. ROC BR have higher sensitivity than prevalence BR 2. The ratio of sensitivity to specificity can, in part, be controlled

18 Conclusion: Correspondence to U.S. MCMI-III The measured constructs are identical: DSM personality disorders and clinical syndromes BUT Millon et al. (1997) considered the ability of the MCMI-III to avoid false positives important (cf. forensic evaluations), whereas priority in the Belgian version is to avoid false negatives (cf. screening).

19 Reference list Butcher, J. (Ed.) (1996). International adaptations of the MMPI-2. Minneapolis, MN: University of Minnesota Press. Derksen, J., & Sloore, H. (2005). Issues in the international use of psychological tests. In S. Strack (Ed.), Handbook of Personology and Psychopathology (pp ). New York: Wiley. Hsu, L.M. (2002). Diagnostic validity statistics and the MCMI-III. Psychological Assessment, 14, Millon, T., Davis, R.D., & Millon, C. (1997). MCMI-III manual (2nd ed.). Minneapolis, MN: National Computer Systems. Millon, T., Millon, C., & Grossman, S. (2006). MCMI-III manual (3rd ed.). Minneapolis, MN: Pearson Assessments. Nurnberg, H.G., Martin, G.A., Somoza, E., Coccaro, E.F., Skodol, A.E., Oldham, J.M., et al. (2000). Identifying personality disorders: Towards the development of a clinical screening instrument. Comprehensive Psychiatry, 41, Rossi, G., & Sloore, H. (2005). International uses of the MCMI: Does interpretation change? In Craig, R. (Ed.). New directions in interpreting the Millon Clinical Multiaxial Inventory (MCMI): Essays on current issues, (pp ). Hoboken, NJ: Wiley. Swets, J.A., Dawes, R.M., & Monahan, J. (2000). Psychological Science can improve diagnostic decisions. Psychological Science in the Public Interest, 1, West, S.G., & Finch, J.F. (1997). Personality measurement: Reliability and validity issues. In R. Hogan, J. Johnson, & S. Briggs (Eds.), Handbook of personality psychology (pp ). San Diego, CA: Academic Press.