Incremental Validity of the MMPI-2 Content Scales in an Outpatient Mental Health Setting

Transcription

1 sychological Assessment 999, Vol., No., 39- Copyright 999 by the American sychological Association, Inc /99/$3 Incremental Validity of the MMI- Content Scales in an Outpatient Mental Health Setting Deanna L. Barthlow, John R. Graham, Yossef S. Ben-orath, and John L. McNulty Kent State University The incremental validity of the Minnesota Multiphasic ersonality Inventory- Content Scales (MMI-; J. N. Butcher, J. R. Graham, C. L. Williams, & Y. S. Ben-orath, 990) was examined using clinical and content scale scores to predict conceptually relevant symptoms and personality characteristics of male and 5 female mental health center outpatients. Regression analyses were performed to determine if the content scales contributed significantly beyond the conceptually relevant clinical scales in predicting therapists' ratings. Of the 0 content scales analyzed, incremental validity was demonstrated for scales for men and 3 scales for women. A nd set of analyses indicated that incremental validity was demonstrated for clinical scales for men and 6 clinical scales for women. The findings provide further evidence that the content scales aid interpretation of MMI- scores by contributing additional information beyond the clinical scales. The clinical scales of the Minnesota Multiphasic ersonality Inventory (MMI; Hathaway & McKinley, 9) were constructed using an empirical-keying approach, resulting in heterogeneity of content for each of the scales. Because not all items of a given scale are related logically to the clinical syndrome that the scale was developed to assess, and the clinical scale score itself does not directly reflect which content areas are endorsed by a test-taker, heterogeneity serves to reduce the ease with which clinical scale scores are interpreted and their meanings are communicated (Butcher, Graham, Williams, & Ben-orath, 990). Consideration of item content became more important as researchers and clinicians recognized its usefulness in score interpretation (Graham, 993). Several attempts have been made to devise an effective method of assessing the item content endorsed by test-takers. One of the earliest approaches involved critical items whose content was judged to be indicative of serious psychopathology (Caldwell, 969; Grayson, 95; Koss & Butcher, 93; Lachar & Wrobel, 99). Another approach was to create content-based subscales within some of the clinical scales (Harris & Lingoes, 955/968). Also attempting to create content scales, Tryon and several colleagues (Stein, 968) used factor analysis to identify clusters of items from across the entire test. In the most thorough and comprehensive effort at creating content scales for the MMI, Wiggins (966) used a combined rational-statistical approach that resulted in 3 content scales formed from across the entire MMI item pool. These scales were Deanna L. Barthlow, John R. Graham, Yossef S. Ben-orath, and John L. McNulty, Department of sychology, Kent State University. Results of this study were presented at the 3nd Annual Symposium on Recent Developments in the Use of the MMI-/MMI-A in Minneapolis, Minnesota. The research was conducted in partial fulfillment of Deanna Barthlow's master of arts degree requirements, under the supervision of John R. Graham. We acknowledge the contributions of Beverly Kaemmer and the University of Minnesota to this research project. Correspondence concerning this article should be addressed to John R. Graham, Department of sychology, Kent State University, Kent, Ohio. Electronic mail may be sent to jgraham@kent.edu. not maintained in the MMI- because they do not represent fully the new content areas that were added. In addition, the Religious Fundamentalism scale could no longer be scored because of item deletions (Graham, 993). With the MMI revision and the subsequent decision not to use the Wiggins content scales came an effort to create new content scales for use with the MMI-. Developed by Butcher et al. (990), the 5 content scales resulted from a five-stage, rationalstatistical approach similar to that used by Wiggins (966). The goal was to specify which MMI- items were related conceptually and statistically to various content areas relevant to the realms of personality and psychopathology. Consideration of content was weighted most heavily in the placement of items on scales, with statistical techniques serving to refine and improve the scales. The scales are internally consistent, are representative of the content areas of the MMI-, and have limited item overlap (Graham, 993). The scale developers provided some initial evidence of external validity using 8 couples from the MMI- normative sample who rated each other on 0 personality and behavior items (Butcher et al., 990). Correlations between scales based on these ratings and content-scale scores yielded behavioral correlates descriptive of high scorers for of the content scales. Subsequent studies have provided additional evidence of the validity of the MMI- content scales in nonclinical samples. In a study of undergraduates that examined the relationship between scores on the Anger (ANG) scale and on several subscales of the State-Trait Anger Expression Inventory (STAXI; Spielberger, 98), the construct validity of the ANG scale was supported (Carr & Graham, 996). Men and women with higher ANG scale scores reported experiencing and expressing angry feelings more frequently than those with lower ANG scale scores. In their study of 65 college students, Schill and Wang (990) reported a positive correlation between ANG scale scores and anger expression and a negative correlation between ANG scale scores and anger control. Additionally, O'Laughlin and Schill (99) reported a positive correlation between ANG scale scores and self-monitored aggression for 6 college students. 39

2 0 BARTHLOW, GRAHAM, BEN-ORATH, AND McNULTY Some researchers have examined the validity of the MMI- Content scales in various clinical samples. Graham, Ben-orath, and McNulty (in press) determined behavioral correlates of the content scales in an outpatient mental health setting. The MMI- scores of,00 outpatients were correlated with several extratest measures, including the Symptom Checklist-90-Revised (SCL- 90-R; Derogatis, 983) and a therapist-rated atient Description Form (DF). Significant correlates were found for of the content scales. Dwyer, Graham, and Ott (99) correlated content-scale scores with symptom ratings for a sample of psychiatric inpatients. As reported by Graham (993), the convergent and discriminant validity of the content scales was supported for the Anxiety (ANX), Health Concerns (HEA), Bizarre Mentation (BIZ), and Depression (DE) scales, the four scales for which construct-relevant ratings were available. In another study of 68 psychiatric inpatients, the - relationship between the content scales and the external criterion of discharge readiness supported the convergent validity of the ANG, Negative Treatment Attitudes (TRT), and HEA scales, as well as the discriminant validity of the ANG and Social Discomfort (SOD) scales (Concepcion, 99). Boone's (99) study of 6 inpatients reported significant correlations between the DE scale and other measures related to depression. In addition, the DE scale differentiated depressed from nondepressed patients. Butcher et al. (990) reported data comparing the Work Interference (WRK) scale scores for psychiatric patients, hospitalized alcoholics, pilot applicants, and active military personnel. All participants were men, and as expected, the pilot applicants had the lowest WRK scale scores. Being in the position of trying to obtain a job, pilot applicants showed the most positive work attitudes. Military personnel scores approximated the mean level of the normative sample, and the scores of alcoholics and psychiatric patients were significantly higher than the normative sample mean. Support for the predictive validity of the TRT scale was provided in a study of 3 male chronic pain patients in which pretreatment TRT scores predicted treatment-related change and posttreatment functioning (Clark, 996). Clark noted that although the content of some items in the TRT scale indicates negative treatment attitudes, its external correlates relate to overall emotional distress. Using 9 male chronic pain patients, Clark (99) found the ANG scale to be homogeneous and a good measure of trait anger and anger externalization. In summary, existing research data provide evidence of the validity of many of the content scales in both clinical and nonclinical samples. In determining the usefulness of any new scale or set of scales, however, it is necessary to establish that the information yielded adds significantly to predictions of relevant behavior (Butcher, Graham, & Ben-orath, 995). If new scales do not contribute information beyond that which is already attained through the use of existing scales, there is little justification for their use. Thus, researchers have also begun to study, specifically, the incremental validity of the content scales. Ben-orath, Butcher, and Graham (99) examined the incremental validity of the content scales in differentiating between 60 inpatients diagnosed with either schizophrenia or major depression. For men, the DE and BIZ scales added significantly to Scales (Depression) and F (Infrequency) in the differential diagnosis. For women, the BIZ scale added incrementally to scales, F, and (sychasthenia). The range of additional variance accounted for by the content scales across the whole sample was 5% to 0% (mdn = 8%). The results from a second set of regression analyses, in which the order of the scales was reversed, indicated that none of the clinical or validity scales added significantly beyond the relevant content scales. The content scales also demonstrated incremental validity for both men and women in a sample of 59 inpatients (Archer, Aiduk, Griffin, & Elkins, 996). The criterion measures were scale scores from a self-report measure, the SCL-90-R (Derogatis, 983), and from two clinician-rated measures, the Global Assessment Scale (Spitzer, Gibbon, & Endicott, 98) and the Brief sychiatric Rating Scale (Overall & Gorham, 96). The content scales added significantly beyond the clinical scales to the prediction of 5 of the 8 criterion measures for men and 6 of the 3 criterion measures for women. Across the entire sample, the additional variance contributed by the content scales ranged from % to 9% (mdn = 3%). Reversing the analyses, the clinical scales had incremental validity for 3 of 8 criterion measures for men and for 8 of 3 criterion measures for women. The additional variance contributed by the clinical scales ranged from % to 0% (mdn = 3%). Incremental validity of the content scales was also examined in a study of 596 college students (Ben-orath, McCully, & Almagor, 993). Scores on several self-report measures of personality and psychopathology, including depression, anger, anxiety, and psychoticism, served as the criterion measures. Correlations were computed between the T scores of each of the MMI- clinical and content scales and the raw scores of each of the eight criterion measures. Incremental validity was examined through the use of hierarchical regression analyses. For each of the eight criterion measures, the content scales had incremental validity over the clinical scales for both men and women. The additional variance accounted for by the content scales ranged from % to 33% (mdn = 0%). Reversing the analyses, the clinical scales had incremental validity over the content scales for five and four of the measures for men and women, respectively. The additional variance accounted for by the clinical scales ranged from % to 5% (mdn = %). The results of studies of the MMI- content scales to date are encouraging and reflect the practical utility of some of the scales. The content scales have been shown to be reliable and valid across several types of settings and are correlated with other variables representing similar content areas. To further justify the use of the content scales, it has been recommended that additional evidence of their incremental validity is needed, because results may vary across studies due to differences in the setting, criterion measures, and question being investigated (Ben-orath et al., 99, 993). Taking into account this recommendation and the design of each of the previous studies in this area, this study addresses two areas in need of elaboration. First, only one of the previous studies used an outpatient setting. Second, past research relied heavily on selfreport measures as opposed to ratings by others who know the patients well. Using a setting and criterion measures not previously used, the current study examined the incremental validity of the MMI- content scales in an outpatient mental health setting by predicting therapists' ratings on a wide variety of their clients' symptoms and behaviors.

3 INCREMENTAL VALIDITY OF THE MMI- CONTENT SCALES articipants Method articipants were outpatient mental health center clients who were part of a larger study of the correlates of MMI- scales and code types (Graham et al., in press). The study setting was a large, urban community mental health center whose clients represented a diverse range of diagnoses and backgrounds. A variety of treatment programs was available, including individual and group therapy, psychiatric consultations, partial hospitalization, dual diagnosis treatment, and specialized programs related tc family or child abuse and shoplifting. The most frequently used approach was individual or outpatient counseling or therapy. The 55-member staff included psychologists, psychiatrists, registered nurses, counselors, social workers, and chemical dependency counselors. This study's sample is part of the sample of,035 men and, women used in the larger study. Of the original sample, 53 men and 9 women did not take the MMI-. Of the 50 men and 8 women who took the MMI-, 0 men and 60 women had valid protocols, and 9 men and 08 women had invalid protocols. The final sample for this study consisted of the men and 5 women who had valid MMI-s and for whom therapist ratings of their clients were available. As compared to the group with invalid MMI-s and the group for which no MMI- was available, the group with valid MMI-s was slightly more educated, had a greater proportion of Caucasian people, was slightly less disturbed, and had a fewer number of previous psychiatric hospitalizations. The three groups did not differ significantly on the number of previous outpatient treatment experiences or the percentage of Axis I and II diagnoses at admission. In comparison with the group with valid MMI-s, this study's sample did not significantly differ on any demographic or mental health status variables. Tables and report the demographic and mental health status characteristics of the sample.' Instruments Intake form. The intake form was developed specifically for the larger study (Graham et al., in press) and was designed to be completed by a trained intake worker on the basis of a personal interview with the client. It includes demographic information, mental health history, substance- Table Demographic Characteristics of Sample Variable Age (years) Years of education Race (%) Caucasian African American/other Marital status (%) Married Widowed Divorced Separated Never married Employment status" (%) Full time art time Unemployed Disabled Other M Men (N = ) SD M Women (N = 5) articipants may be in more than one employment status category. SD 0..0 Table Mental Health History and Status Characteristics of Sample Men Women (N = ) (N = 5) Variable M SD M SD Current level of functioning (Axis V) revious psychiatric hospitalization (%) revious outpatient treatment (%) Current medications (%) Antipsychotic Antidepressant Lithium Antianxiety Other Any Axis I diagnosis (%) Specific Axis I diagnosis (%) Adjustment disorder Depression Anxiety disorders Substance abuse/dependence Any Axis II diagnosis (%) abuse history, diagnostic impressions for all five axes of the Diagnostic and Statistical Manual of Mental Disorders (DSM III R; American sychiatric Association, 98) and ratings on mental status variables such as orientation, memory, and mood. Demographic and mental health status information for the sample was obtained from the intake form. MMI-. The MMI- (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 989) is a 56-item personality inventory that is the revised and updated version of the original MMI. It includes the validity and clinical scales from the original MMI, some new validity scales developed specifically for the MMI-, content scales, content component scales, and other supplementary scales. Norms were developed for the MMI- that were representative of the population of the United States. sychometric characteristics of the MMI- scales have been summarized in its manual (Butcher et al., 989). The MMI- content scales have internal consistencies ranging from. to.9 for clinical samples and test-retest reliabilities ranging from.8 to.9 in the normative sample (Butcher et al., 990). Item overlap between the content scales and the clinical scales varies from one content scale to another. Uniform T scores are used so that content-scale scores can be compared with each other and with clinicalscale scores (Graham, 993). atient Description Form. The DF was developed specifically for the larger study (Graham et al., in press). 3 Its 88 items permit ratings of personality and symptomatic characteristics of patients using a 5-point scale (not at all, slight, moderate, high, and very high). The DF was constructed so therapists could provide descriptions of their clients. Items for the DF were selected on the basis of a review of previous research and current interpretive reference sources. The items represent commonly identified symptoms and characteristics that are correlates of MMI- clinical scales and code types. Using a combined rational-statistical approach, the authors developed 5 scales to assess the major content dimensions of the DF. The internal consistency coefficients for the DF scales, based on data from the clients who participated in the larger study, A copy of the demographic, mental health history, and mental health status characteristics of the valid MMI-, invalid MMI-, and no MMI- groups is available from John R. Graham. A copy of the Intake Form is available from John R. Graham. 3 A copy of the atient Description Form (DF) and a copy of the internal consistency coefficients for the DF scales used in this study are available from John R. Graham

4 BARTHLOW, GRAHAM, BEN-ORATH, AND McNULTY ranged from.69 to.9 for men (mdn =.8) and from. to.93 for women (mdn =.8). The intercorrelations of the DF scales ranged from. to. for men (mdn =.) and from. to. for women (mdn =.). ractical limitations within the setting precluded the possibility of collecting interrater reliability and temporal stability data. Analyses of the internal consistency of the DF scales and of the correlations of the DF scales with the MMI- clinical and content scales indicated that raters were consistent in their use of this instrument both across and within professions. rocedure ersons requesting services at the mental health center participated in extensive intake interviews conducted by experienced professional mental health workers, mainly social workers and nurses, who had been trained by research project staff on how to complete the intake form. After the intake interviews, clients were scheduled to complete the MMI- and additional measures not included in this study. The median time between the intake interview and administration of the MMI- was days. The MMI-s were administered to clients individually or in small groups. A taperecorded version of the MMI- was used with participants who had limited reading ability. Clients were assigned to therapists and treatment programs after completing the intake interview. Therapists completed the DF and several other measures not used in this study after three treatment appointments had been kept. The median length of time between MMI- administration and DF completion was 38 days. Therapists completed the DF before gaining access to any MMI- results. The only exceptions were in a few cases (less than 3%) in which the computerized MMI- interpretations indicated issues that warranted special attention, such as suicide potential. In these instances, psychologists reviewing the interpretations informed the therapists of the specific issue but did not share any MMI- scores or further interpretations with them. MMI- protocols were considered valid if (a) 30 or less items were omitted, (b) Variable Response Inconsistency (VRIN) scale T scores were less than or equal to 80, (c) True Response Inconsistency (TRIN) scale raw scores were between 6 and, (d) F scale raw scores were less than for men and less than 9 for women, and (e) Fb scale raw scores were less than 3 for men and less than for women. For each of the 5 content scales, an attempt was made to identify rationally one or more DF scales that were conceptually related to that particular content scale. One or more clinical scales were then rationally identified as being conceptually related to each of the chosen DF scales. Thus, a content scale was included in the analysis for incremental validity if at least one DF scale was identified as being conceptually related to that content scale and if at least one clinical scale was judged to be conceptually related to that DF scale. Through this rational approach, one DF scale was identified for each of 0 of the content scales. One clinical scale was identified for each of those DF scales, except in the case of the DF scale, for which two clinical scales appeared to be conceptually related. No validity scales were considered when creating scale pairs in light of the recent suggestion that the clinical and content scales are better suited for clinical inferences and that the validity scales should be used in reference to test-taking attitude only (Graham et al., in press). The scale combinations are reported in Table 3. The procedure for deciding which scales to include in the analyses differs from that used in the three previous incremental validity studies, in which all of the clinical and content scales included in the study were entered as blocks into the prediction of each dependent variable. In the previous studies, the incremental contribution of the content scales was examined in light of the best empirical prediction of each dependent variable. The question in this study was whether a given content scale would add a significant amount of unique variance to a given clinical scale, the item content of which is conceptually relevant to that of the content scale with which it is paired. In other words, given two scales somewhat similar in item content and in the construct each measures, the goal was to determine if the content scale (developed using a rational-statistical ap- Table 3 Scale Combinations Used in Regression Analyses MMI- Content Scale Anxiety Depression Health Concerns Bizarre Mentation Anger ractices ractices Low Self-esteem Social Discomfort Family roblems Obsessiveness DF Scale sychotic Symptoms Family roblems MMI- Clinical Scale Note. MMI- = Minnesota Multiphasic ersonality Inventory-; DF = patient description form. = Hypochondriasis; = Depression; = sychopathic Deviate; = sychasthenia; 8 = Schizophrenia; 9 = Hypomania; 0 = Social Introversion. proach) would contribute incrementally to the clinical scale (constructed using an empirical approach) in the prediction of patients' relevant behavior. Results All analyses were performed separately for men and women because there has been some suggestion in the literature that there are differences in MMI- correlates on the basis of gender (Graham et al., in press; Schinka, LaLone, & Greene, 998). Mean scores and standard deviations were computed for each of the MMI- clinical and content scales and for each of the DF scales used in the analyses. For each scale combination, the clinical and content scales were each correlated with the DF scale and with each other. 5 To test the incremental validity of the content scales, we calculated hierarchical regression analyses separately by gender for each of the 0 content scales used in this study. In each analysis, the conceptually relevant DF scale served as the criterion variable, and the MMI- clinical and content scales that were judged to be conceptually relevant to that DF scale served as the predictor variables. Two clinical scales were identified as conceptually related to the DF scale, so the AS content scale was used twice. Thus, different combinations of scales were used. For each combination, the clinical scale was entered by itself as the first block in the prediction of the DF scale, followed by the content scale in the second block. The change in variance accounted for from Block to Block 'served as a test of the incremental validity of the content scale. Because each equation had only one dependent variable, the DF scale, alpha was set at p < with no correction applied. The number of cases in each equation varies slightly because of missing DF data. Of the 0 different content scales used in this study, incremental validity was shown for scales for men and for 3 scales for women. The sizes A copy of the mean T scores and standard deviations for the MMI- Clinical and Content scales and for the DF scales is available from John R. Graham. 5 A copy of the correlations among the DF scales, MMI- Clinical Scales, and MMI- Content Scales is available from John R. Graham

5 INCREMENTAL VALIDITY OF THE MMI- CONTENT SCALES 3 of the variance increments were modest in each case. The amount of additional variance accounted for by the content scales ranged from % to 6% for men and from % to % for women. Tables and 5 report the results of the analyses for men and women, respectively. In a second set of analyses, the entry sequence of the clinical and content scales was reversed to test the incremental validity of the clinical scales. Of the different clinical scales used in the reverseorder analyses, incremental validity was shown for scales for men and for 6 scales for women. The amount of additional variance accounted for by the clinical scales ranged from % to 8% for men and from % to 3% for women. Tables 6 and report the results of the analyses for men and women, respectively. Discussion The results of this study indicate that many of the MMI- content scales demonstrated incremental validity in the prediction of therapists' ratings of clients' behavior and personality characteristics. Specifically, incremental validity was demonstrated for the ANX, DE, HEA, ANG, AS, LSE, and FAM content scales for men, and the BIZ, AS, and FAM content scales for women. Although the amounts of variance contributed by the content scales were modest in size in this study, this new information can only aid in our understanding of outpatients. Small increments in knowledge can be beneficial as long as the cost to obtain those increments does not outweigh their benefits. Because scores and clinical interpretations based on them are readily available for both the clinical and the content scales, it is advisable to integrate and apply the information gained from both. Thus, these results provide further support for the use of some of the content scales. This study was conducted in an outpatient psychiatric setting, using a criterion measure of therapists' ratings of patients' personality characteristics and psychopathology. Although this study's setting and criterion measure differ slightly from those of previous studies, the results are consistent. By concluding that many of the content scales contribute additional, unique information beyond that which is provided by the clinical scales, this study is in agreement with the three previous studies in this area. Ben-orath et al. (99) found that DE and BIZ contributed incrementally to the clinical scales in the prediction of clinician diagnosis. Archer et al. (996) found evidence of the incremental validity of all of the content scales except TRT in a sample of inpatients using self- Table Regression Analysis Results for the Content Scales Men f test F chg analysis DF Scale/Block sychotic Symptoms Family roblems MMI- Scale, ANX, DE, HEA 8 8,, ANG, AS 9 9, AS, 0 0,,, BIZ LSE SOD FAM OBS R R r> "Adj Note. R^ = R adjusted: an estimate of the correlation in the population (Howell, 99); DF = patient description form: MMI- = Minnesota MuJtiphasic ersonality Inventory-; ANX = Anxiety; DE = Depression; HEA = Health Concerns; BIZ = Bizarre Mentation: ANG = Anger; AS = ractices; LSE = Low Self-Esteem; SOD = Social Discomfort; FAM = Family roblems; OBS = Obsessiveness. ^ F df,9,8,,6,8,,55,5,55,5,3,36,3,36,9,8,68,6,,,3,3 6 ^ch

6 BARTHLOW, GRAHAM, BEN-ORATH, AND McNULTY Table 5 Regression Analysis Results for the Content Scales Women Ftest F chg analysis DF Scale/Block MMI- Scale R R R Adj R F df F chg sychotic Symptoms Family roblems, AWX, > >, HEA 8 8, BIZ, AWG, AS 9 9, AS, Z,S 0 0, SOD, FAM, OBS ,39,393,395,39,385,38,0,0,3,3,38,3,38,3,398,39,09,08,39,390,3, Note. R^ = R adjusted: an estimate of the correlation in the population (Howell, 99); DF = patient description form; MMI- = Minnesota Multiphasic ersonality Inventory-; ANX = Anxiety; DE = Depression; HEA = Health Concerns; BIZ = Bizarre Mentation; ANG = Anger; AS = ractices; LSE = Low Self-Esteem; SOD = Social Discomfort; FAM = Family roblems; OBS = Obsessiveness. report and clinician-rated measures. Finally, Ben-orath et al. (993) found that all of the content scales except BIZ, FAM, AS, and WRK, contributed incrementally to the prediction of several self-report measures in a sample of college students. The additional variance accounted for by the content scales in this study was similar in size to that of Archer et al. (996) and Ben-orath et al. (99), and it was slightly smaller than that of Ben-orath et al. (993). The latter finding may be because, in part, of the use of clinicians' ratings as criterion measures in this study as opposed to the use of self-report measures in Ben-orath et al. (993). That findings for individual content scales were not exactly the same across all four studies reflects the differences among those studies in purpose, setting, and criterion measures. When drawing specific conclusions concerning incremental validity in this study, there are two main issues to take into account: (a) the similarity of the constructs measured by the DF, clinical, and content scales in a given combination, and (b) the range of symptoms, behaviors, and personality characteristics represented in the sample. First, because the scales were chosen for analysis through rational judgment, it is possible that all three scales in a given combination may be measuring slightly different constructs, and therefore, neither the clinical nor the content scale could contribute incremental information beyond the other to the prediction of the DF scale. Furthermore, in cases in which the clinical and content scales are highly correlated and each accounts for almost the same amount of variance in the prediction of the DF scale, the content scale might not contribute incrementally. For example, the SOD scale did not add beyond Scale 0 to the prediction of the DF scale for either men or women. The correlation between the SOD scale and Scale 0 is.8 for men and.89 for women. Consequently, there is very little variance specific to the content scale that could serve to improve the prediction of the DF scale beyond the clinical scale. A likely reason for this lack of variance may be due to item overlap between the SOD scale and Scale 0. Eighteen of the total items on the SOD scale overlap with those on Scale 0, which has a total of 69 items. The pairing of Scale and the HEA scale is the only other in which a large amount of item overlap exists. The correlation between the clinical and content scales for this scale combination is also quite high at.85 for men and.9 for women, possibly explaining why little to no incremental contribution was found for either scale. Second, the scores on several DF scales were somewhat restricted in range because of the nature of the sample and, thus, may

7 Table 6 Regression Analysis Results for the Clinical Scales Men INCREMENTAL VALIDITY OF THE MMI- CONTENT SCALES 5 F test F chg analysis DF Scale/Block MMI- Scale R R «d R F df F chg sychotic Symptoms Family roblems ANX ANX, DE DE, HEA HEA, BIZ BIZ, 8 ANG ANG, AS AS, AS AS, 9 LS is, SOD SOD, 0 FAM FAM, OBS OBS, ,9,8,,6,8,,9,9,55,5,3,36,3,36,9,8,68,6,,,3, Note. /?a dj = R adjusted: an estimate of the correlation in the population (Howell, 99); DF = patient description form; MMI- = Minnesota Multiphasic ersonality Inventory-; ANX = Anxiety; DE = Depression; HEA = Health Concerns; BIZ = Bizarre Mentation; ANG = Anger; AS = ractices; LSE = Low Self-Esteem; SOD = Social Discomfort; FAM = Family roblems; OBS = Obsessiveness. have attenuated correlations between the DF scales and the clinical and content scales. For example, because the community mental health center used in this study refers patients exhibiting psychotic symptoms to other agencies, the sychotic Symptoms DF scale scores reflect very little psychotic behavior in the sample. To better understand the relationship between Scale 8 and BIZ in predicting psychotic behavior, patients displaying these behaviors should be better represented in the sample. Although the focus of this study is on the content scales, reverse order analyses were performed to determine the incremental validity of the clinical scales. All of the clinical scales were included in this study except for Scales 3, 5, and 6. The results indicated that several of the clinical scales contributed significant additional variance beyond the content scales. Specifically, incremental validity was demonstrated for Scales,,, and for men and Scales,,,, 9, and 0 for women. Several gender differences emerged from the analyses. As noted earlier, data were analyzed separately by gender for this study because recent investigations have pointed to gender differences in the empirical correlates of MMI- clinical and content scales. These findings are consistent with this trend in the literature and underscore the importance of considering gender in interpreting MMI- scores. It is unclear exactly why gender differences in incremental validity occurred in this study. Future investigations should focus on determining the reasons for these differences and on elaborating their implications. In some scale combinations, only the content scale showed incremental validity, whereas in other cases, only the clinical scale contributed significant additional variance. Finally, in some cases, both the clinical and content scales in a scale combination added incrementally. One might be concerned that the content scales may have been spuriously favored by the fact that both they and the DF scales were created by using a rational-statistical approach to scale construction, whereas the clinical scales were constructed by using an empirical approach. However, because each DF item reflects an empirical correlate of of the 0 clinical scales, it is unlikely that the content scales would be at any advantage over the clinical scales in demonstrating incremental validity. That both the clinical and content scales demonstrate incremental validity is evidence that the two types of scales account for some unique variance in predictions of behavior and personality characteristics. These findings provide further support for the use of both the clinical and content scales in clinical interpretation rather than either one of them alone. In addition to considering the

8 6 BARTHLOW, GRAHAM, BEN-ORATH, AND McNULTY Table Regression Analysis for the Clinical Scales Women F test F chg analysis DF Scale/Block MMI- Scale R R R R F df F chg sychotic Symptoms Family roblems ANX ANX, DE DE, HEA HEA, BIZ BIZ, 8 A j AMJ, AS AS, AS AS, 9 LS LSE, SOD SOD, 0 FAM FAM, OBS OBS, ,39,393,395,39,385,38,0,0,3,3,38,3,38,3,398,39,09,08,39,390,3, Note. Rl Aj = R adjusted: an estimate of the correlation in the population (Howell, 99); DF = patient description form; MMI- = Minnesota Multiphasic ersonality Inventory-; ANX Anxiety; DE = Depression; HEA = Health Concerns; BIZ = Bizarre Mentation; ANG = Anger; AS = ractices; LSE = Low Self-Esteem; SOD = Social Discomfort; FAM = Family roblems; OBS = Obsessiveness. statistical significance of the results, though, it is also important to examine the findings in terms of clinical meaningfulness. There is no accepted standard regarding how much new information a scale should provide in order to be judged clinically useful. Individual researchers and clinicians must ask themselves if the amount of unique information contributed by the content scales regarding test-takers' behavior and personality characteristics is sufficient to warrant their usage. However, one could argue that regardless of amount, any unique information contributed by the content scales brings clinicians closer to the goal of increasing their overall understanding of their clients. As discussed earlier, the benefits of using both the clinical and the content scales outweigh the costs given that scores for both sets of scales are included on a standard profile and that interpretive material is readily available for both sets. Because of their facevalid nature, the content scales serve to facilitate communication between the test-takers and those interpreting the results. Because elevations on the clinical scales may be due to one or more causes because of the heterogeneity of their item content, the content scales can also serve as a source for ruling out interpretive hypotheses suggested by the clinical scales. A particularly interesting scenario arises in the case where the clinical scales are within normal limits, but one or more of the content scales are elevated (assuming a valid profile). Thus, the content scales are particularly helpful in clarifying convergent information provided by the clinical scales, in discriminating between alternative hypotheses obtained from the clinical scales, and in providing interpretations beyond those suggested by the clinical scales. By using both sets of scales, then, we gain a clearer and more accurate understanding of test-takers. In terms of integrating the information obtained from the two sets of scales, the scale scores need to be considered individually in order to draw interpretations from the research base on scale correlates. These separate interpretations can then be combined in a rational, rather than empirical, manner when making clinical inferences about a test-taker. Future studies should incorporate criterion measures that assess constructs other than the more commonly included ones of depression, anxiety, and anger. More knowledge of such scales as LSE, AS, SOD, WRK, and TRT could be gained by including criterion measures that are specifically and conceptually related to these particular content scales. Utilizing conceptually relevant extratest measures to examine these scales in more detail may lead to more definitive conclusions about their usefulness in providing unique information to predictions of behavior.

9 INCREMENTAL VALIDITY OF THE MMI- CONTENT SCALES This study adds to the growing body of literature indicating that the content scales are a useful addition to the clinical scales in MMI- interpretation. As this topic continues to be examined further, a clearer picture will be gained of how much incremental variance the content scales add, which content scales consistently contribute more or less than others, and how the contributions vary across types of populations and criterion measures, thus clarifying for mental health professionals the role of the content scales in clinical practice. References American sychiatric Association. (98). Diagnostic and statistical manual of mental disorders (3rd ed., rev.). Washington, DC: Author. Archer, R.., Aiduk, R., Griffin, R., & Elkins, D. E. (996). Incremental validity of the MMI- content scales in a psychiatric sample. AM^OTment, 3, Ben-orath, Y. S., Butcher, J. N., & Graham, J. R. (99). Contribution of the MMI- content scales to the differential diagnosis of schizophrenia and major depression. sychological Assessment, 3, Ben-orath, Y. S., McCully, E., & Almagor, M. (993). Incremental validity of the MMI- content scales in the assessment of personality and psychopathology by self-report. Journal of ersonality Assessment, 6, Boone, D. E. (99). Validity of the MMI- depression content scale with psychiatric inpatients. sychological Reports,, Butcher, J. N., Dahlstrom, W. G., Graham, J. R., Tellegen, A., & Kaemmer, B. (989). Minnesota Multiphasic ersonality Inventory-: Manual for administration and scoring. Minneapolis: University of Minnesota ress. Butcher, J. N., Graham, J. R., & Ben-orath, Y. S. (995). Methodological problems and issues in MMI, MMI-, and MMI-A research. sychological Assessment,, Butcher, J. N., Graham, J. R., Williams, C. L., & Ben-orath, Y. S. (990). Development and use of the MMI- content scales. Minneapolis: University of Minnesota ress. Caldwell, A. B. (969). MMI critical items. Unpublished manuscript. Carr, J. L., & Graham, J. R. (996). Assessing anger with the Minnesota Multiphasic ersonality Inventory. In C. D. Spielberger, I. G. Sarason, J. M. T. Brebner, E. Greenglass,. Laungani, & A. M. O'Roark (Eds.), Stress and emotion: Anxiety, anger, and curiosity (pp. 6-80). Washington, DC: Taylor and Francis. Clark, M. E. (99). Interpretive limitations of the MMI- anger and cynicism content scales. Journal of ersonality Assessment, 63, Clark, M. E. (996). MMI- Negative Treatment Indicators content and content component scales: Clinical correlates and outcome prediction for men with chronic pain. sychological Assessment, 8, Concepcion, E. E. (99). Validity of the MMI- content scales with psychiatric inpatients. Unpublished doctoral dissertation, Kent State University, Kent, Ohio. Derogatis, L. R. (983). SCL-90-R: Administration, scoring, and procedures manual-h. Towson, MD: Clinical sychometric Research. Dwyer, S. A., Graham, J. R., & Ott, E. K. (99). sychiatric symptoms associated with the MMI- content scales. Unpublished manuscript, Kent State University, Kent, Ohio. Graham, J. R. (993). MMI-: Assessing personality and psychopathology (nd ed.). New York: Oxford. Graham, J. R., Ben-orath, Y. S., & McNulty, J. L. (in press). MMI- correlates for outpatient mental health settings. Grayson, H. M. (95). sychological admission testing program and manual. Los Angeles: Veterans Administration Center, Neuropsychiatric Hospital. Harris, R. E., & Lingoes, J. C. (955/968). Subscales for the MMI: An aid to profile interpretation. Unpublished manuscript. Hathaway, S. R., & McKinley, J. C. (9). The Minnesota Multiphasic ersonality Inventory. Minneapolis: University of Minnesota ress. Howell, D. C. (99). Statistical methods for psychology (th ed.). Belmont, CA: Wadsworth ublishing Company. Koss, M.., & Butcher, J. N. (93). A comparison of psychiatric patients' self-report with other sources of clinical information. Journal of Research in ersonality,, Lachar, D., & Wrobel, R. A. (99). Validating clinicians' hunches: Construction of a new MMI critical item set. Journal of Consulting and Clinical sychology,, -8. O'Laughlin, S., & Schill, T. (99). The relationship between selfmonitored aggression and the MMI- F,, 9 composite and anger content scale scores. sychological Reports,, Overall, J. E., & Gorham, D. R. (96). The Brief sychiatric Rating Scale. sychological Reports, 0, Schill, T., & Wang, S. (990). Correlates of the MMI- Anger Content Scale. sychological Reports. 6, Schinka, J. A., LaLone, L., & Greene, R. L. (998). Effects of psychopathology and demographic characteristics on MMI- scale scores. Journal of ersonality Assessment, 0, 9-. Spielberger, C. D. (98). State-Trait Anger Expression Inventory manual. Odessa, FL: sychological Assessment Resources. Spitzer, R. L., Gibbon, M., & Endicott, J. (98). Global Assessment Scale (GAS). New York: Biometrics Research. Stein, K. B. (968). The TSC scales: The outcome of a cluster analysis of the 550 MMI items. In. McReynolds (Ed.), Advances in psychological assessment, Vol. I (pp. 80-0). alo Alto, CA: Science and Behavior Books. Wiggins, J. S. (966). Substantive dimensions of self-report in the MMI item pool. sychological Monographs, 80 (, Whole No. 630). Received April 30, 998 Revision received October 5, 998 Accepted October 9, 998