Measuring Interpersonal Problems in People With Mental Retardation

Transcription

1 Measuring Interpersonal Problems in People With Mental Retardation Stephen Kellett, Nigel Beail, and David W. Newman Keresforth Centre (Barnsley, UK) Abstract Despite interpersonal problems being commonplace in the clinical presentations of people with mental retardation, previous efforts to index interpersonal difficulties have tended to unsatisfactorily rely on external ratings. The Inventory of Interpersonal Problems-32 is a psychometrically robust self-report measure of interpersonal problems in adult populations. We found that this instrument retained many of its psychometric properties when used with a sample of 255 respondents with mild mental retardation. Results indicate four usable subscales alongside the full-scale when employed with this population. The subscales produced generally satisfactory internal and test retest reliabilities and sound concurrent and internal external criterion-related validities. Results are discussed with reference to further development of the measure in relation to people with mental retardation. Evaluators of psychotherapeutic interventions with people who have mental retardation have tended to rely exclusively on externally rated behavior in terms of psychotherapy outcome (Beail, 2003; Prout & Nowak-Drabik, 2003). This type of focus, however, leads to accusations of overly narrow and constrictive definitions of outcome. There are no known measures solely focused on the clients perceptions of their functioning and associated difficulties in the interpersonal domain. Kellett, Beail, Newman and Hawes (2004) stated that the development of such measures of interpersonal difficulties would widen both the field of clinical enquiry into the psychological distress reported by clients at assessment and also the breadth of measures available for intervention outcome studies (p. 280). The range of psychological interventions available (e.g., psychodynamic psychotherapy, cognitive behavioral psychotherapy) all contain strong interpersonal features, due to the pivotal and ubiquitous nature and influence of the therapeutic relationship (Roth & Fonagy, 1996). Psychotherapeutic intervention may alter typically dysfunctional methods of relating and interacting as a means of increasing psychological well-being and mental health. The use of self-report measures with people who have mental retardation is methodologically and clinically challenging due to problems that arise in relation to item content, question phrasing, bandwidth fidelity, response format, and general psychometric properties/issues (Bramston & Fogarty, 2000; Finlay & Lyons, 2001). However, recent studies of the psychometric properties of single and multi-trait symptom scales, used in conjunction with people who have mental retardation, illustrate that this is an evolving area of research and clinical interest (Kellett, Beail, Newman, & Frankish, 2003; Kellett et al., 2004; Kellett, Beail, Newman, & Mosley, 1999; Mindham & Espie, 2003; Powell, 2003). The impact of interpersonal problems in the lives of people with mental retardation has, therefore, not been subject to intensive or continued scientific enquiry to date. Ethically, this is troubling, as people who have mental retardation undoubtedly exist in the same interpersonal milieu as people in the general population. This situation was interestingly mirrored within the general population, with a similar paucity of information per- 136 American Association on Mental Retardation

2 taining to people s experiences of their interpersonal patterns and associated mental health problems. This impasse was resolved in 1988 with the development of the Inventory of Interpersonal Problems, an easily administered, 127 item, selfreport inventory of interpersonal difficulties (Horowitz, Rosenberg, Baer, Ureno, & Villasenor, 1988). The inventory was designed to identify the most common interpersonal problems that psychiatric patients experience and subsequently enable the measurement of any change due to intervention. Measures with an interpersonal focus also assist in efforts to differentiate that distress caused by interpersonal problems in contrast to the distress created by noninterpersonal problems, such as unwanted thoughts or obsessive ruminations (Horowitz et al., 1988). Subsequent factor analytic studies pruned the item count of the Inventory to produce a more compact, but valid and reliable, 32-item version (Barkham, Hardy, & Startup, 1996). Beail and Warden (1996) reported in their pilot work that the Inventory of Interpersonal Problems-32 could be used in an interview format with some patients who have mild mental retardation. Beail (2000) then used it as a measure in a psychotherapy outcome study. However, the psychometric foundations of the Inventory of Interpersonal Problems-32 were not examined in such studies, undermining the credibility of the results. In this paper we report on a study of various aspects of reliability and validity of the Inventory of Interpersonal Problems-32 with adults who have mild mental retardation. Our purpose was to increase the range of options open to clinicians and researchers regarding the assessment of interpersonal problems. Method Participants Participants were 255 people with mental retardation. Their average age was years (standard deviation [SD] 12.09), ranging from 16 to 66 years. The sample consisted of 162 males with an average age of years (SD 12.88) and 93 females, whose average age was years (SD 10.71). All participants were either already diagnosed or were in the process of being diagnosed as having mild mental retardation. Within the sample, two distinct study groups were recruited, which are described below. The first group (called the community group in the subsequent text and tables) consisted of people living in the community who had been referred for intellectual assessment. They had no apparent psychological, behavioral, or interpersonal distress. Only those individuals who met the American Association on Mental Retardation (Luckasson et. al., 2002) diagnosis for mental retardation were included in the community group. This group consisted of 98 participants of whom 55 were male (56.12%), with an average age of years and 43 were female (43.88%), with an average age of years. The second group (called the clinical group in the subsequent text and tables) contained 157 participants, 106 males (67.51%) with an average age of years and 51 females (32.49%), with an average age of 31 years. The clinical group had previously been assessed as having mental retardation but had on this occasion been referred due to apparent psychological/behavioral/emotional distress and assessment for potential psychological intervention. Reasons for referral fell into the following diagnostic groupings: behavior disorder (57.3%), mood disorder (15.9%), anxiety disorder (10.2%), problems related to abuse (5.7%), psychotic disorder (4.5%), somatoform disorder (1.9%), eating disorder (1.9%), substance-related disorder (1.9%), and sleep disorder (0.6%). Measures The Inventory of Interpersonal Problems-32 is a 32-item self-report questionnaire designed to index the difficulties adults typically experience in their interpersonal relationships. This inventory is the result of factor analysis of the Inventory of Interpersonal Problems-127 (Horowitz et al., 1988) designed to provide a short, psychometrically robust measure of interpersonal difficulties. The Inventory of Interpersonal Problems-32 contains items that are equally divided to reflect interpersonal skills that people may find too hard to do (e.g., join in on groups) or responses that they do too much (e.g., get irritated). Items are scored on a 5-point scale from 0 (not at all) to4 (extremely). Factor analytic evidence in adult populations suggests that this Inventory forms eight 4-item subscales: Hard to be Sociable, Hard to be Assertive, Hard to be Supportive, Hard to be Involved, Too Aggressive, Too Open, Too Caring, and Too Dependent (Barkham et al., 1996). The Inventory of Interpersonal Problems-32 was completed along with the Brief Symptom Inventory (Derogatis, 1993), which contains an Interpersonal Sensitivity subscale that according to its devel- American Association on Mental Retardation 137

3 oper is a measure of marked discomfort during interpersonal interactions. This subscale has been found to have good internal reliability and discriminative validity when employed with people who have mild mental retardation (Kellett et al., 2003). Procedure Established instructions for administering the Inventory of Interpersonal Problems-32 are for assessment of nondisabled adults. Kellett et al. (1999) illustrated the efficacy of an assisted completion format (pp ) for completion of self-report psychometric measures with people who have mild mental retardation. They provided instructions for psychometric administration that were replicated and followed in the current study. The assessment was completed in a one-to-one format in a private setting. Items were read to the participant verbatim and in sequence from the Inventory of Interpersonal Problems-32 and Brief Symptom Inventory score sheet and manual. An answer feedback form was employed to aid participants responses through providing the five possible responses in the form of numerical (0 to 4), grammatical (not at all to a lot) and pictorial representations (sad face to happy face). This form is reproduced in the appendix of Kellett et al. (1999). Results The construct validity of the Inventory of Interpersonal Problems-32 was investigated via exploratory factor analysis. Bartlett s test of sphericity was highly significant, p.001, indicating that the data were suitable for factor analysis. The principal component analysis results of the Inventory of Interpersonal Problems-32 are presented in Table 1, for which we employed the Kaiser-Guttman criterion (Guttman, 1954) requiring eigenvalues to be greater than 1.00 for factor retention. Eight interpretable factors were derived from a normal varimax rotation of the principal components (Harman, 1967), with the factors accounting for 58.87% total of the variance in the matrix. A detailed representation of the factor loadings appears in Table 2. The factor analysis results have been organized in comparison to the original factor solution in Table 2 in order to appraise replication with a sample of adults with mental retardation. The eight-factor solution displays a degree of Table 1. Results of the Principal Components Analysis Factor Eigenvalue %of variance Cumulative % overlap with the original Inventory of Interpersonal Problems-32 factor structure, indicating some amount of factor stability, but also some key differences with the extant structure. The Hard to be Assertive, Hard to be Supportive, and Too Aggressive subscales were perfectly replicated. The original Hard to be Sociable and Hard to be Involved subscale items factored together in the current analysis to form a single scale tapping interpersonal difficulties with being Involved and Sociable with others. The differences occurred with the Too Open, Too Caring, and Too Dependent subscales. Due to similarities and differences emerging between the extant structure and the factor findings with adults who have mental retardation, we conducted subsequent analyses using both the extant subscale structure and the specific factor structure found with adults who have mental retardation (i.e., the Hard to be Involved and Sociable, Hard to be Assertive, Hard to be Supportive, and Too Aggressive subscales). Because the Inventory of Interpersonal Problems-32 has not previously been employed in the assessment of interpersonal problems experienced by people with mental retardation, a major concern centers on the reliability of the subscales. Table 3 contains the reliability information for the full and subscale scores. We performed internal consistency reliability estimates using Cronbach s coefficient alpha (Cronbach, 1951), which is a multipoint variation of the Kudor-Richardson formula (Nunnally, 1970). In order to increase the sensitivity of the reliability observations, we performed reliability analyses on the total sample, the community sample, and the clinical samples. This Inventory was also re-administered to 23 respondents in the community sample 2 weeks following 138 American Association on Mental Retardation

4 initial completion. We assessed the test retest reliability of the full and subscale scores using intraclass correlations (Shrout & Fleiss, 1979), which were all statistically significant. Full-scale Inventory of Interpersonal Problems-32 reliability scores indicate sufficient internal scale consistency across the total, clinical, and community samples. Results illustrate that the four Hard to be subscales and the Too Aggressive subscale produced satisfactory reliability results. Using the specific factor structure found for adults with mental retardation, we found that the Hard to be Involved and Sociable subscale had a test retest reliability of.82, with an alpha of.84 in the total sample and.85 and.82 in the clinical and community samples, respectively. The Too Caring and Too Dependent subscales produced lower reliability results, with the Too Open subscale producing unsatisfactory reliability scores across the total, community, and clinical groups. The full-scale Inventory of Interpersonal Problems-32 and all subscales indicated satisfactory to high stability over time. Nineteen of the individual items produced significant test retest correlations, with significant item-specific correlations ranging from.42 (Item 32) to.85 (Item 8). We tested concurrent validity using Pearson correlation coefficients (r) between the Interpersonal Sensitivity subscale of the Brief Symptom Inventory and the full and subscale Inventory of Interpersonal Problems-32 scores. All subscale correlations were statistically significant, ranging from r.15, p.05, for the Too Open subscale to r.45, p.01, for the Hard to be Assertive subscale. Using the specific factor structure for adults with mental retardation, we found that the Hard to be Involved and Sociable subscale were correlated, r.49, p.01. The full-scale Inventory of Interpersonal Problems-32 score correlated with the Interpersonal Sensitivity subscale, r.55, p.01. Of all the nine Primary Symptom subscales of the Brief Symptom Inventory, the Interpersonal Sensitivity subscale correlated most highly with the Inventory of Interpersonal Problems-32 full-scale score. In testing the internal criterion-related validity (Aiken, 1996), we found that all item:subscale correlations (range.35 to.66) and item:full-scale correlations (range.12 to.74) were statistically significant or highly statistically significant. For the purpose of external criterion-related validity (Aiken, 1996), the presence of the mental health referral to clinical psychology services was taken as the external criterion for the presence of interpersonal problems. The ability of the Inventory of Interpersonal Problems-32 and its constituent subscales to discriminate between the two study groups was investigated by means of independent samples t tests. As can be seen in Table 4, the full-scale Inventory of Interpersonal Problems-32 and the Too Aggressive subscale appear to effectively discriminate between the community and clinical study groups. Using the specific factor structure for adults with mental retardation, the Hard to be Involved and Sociable subscale did not effectively discriminate between the study groups. The range in the clinical group on the full-scale Inventory of Interpersonal Problems-32 score was 0.00 to 3.25 and for the community group, 0.00 to Discussion Our aim in this paper was to examine the psychometric properties of the Inventory of Interpersonal Problems-32 in people with mild mental retardation in order to assess the inventory s viability in terms of clinical assessment, the audit and evaluation of routine clinical practice, and potential research evaluation applications. The various types of reliability and validity illustrated that this instrument may be a useful addition to the assessment of interpersonal problems experienced by people with mild mental retardation. Results of present study illustrate that the construct validity of the Inventory of Interpersonal Problems-32, when employed with people who have mild mental retardation, shares some similarities to those previously illustrated in adults with mental health problems (Barkham et al., 1996). The factor structure extracted from the data indicate that three of the eight subscales were perfectly replicated, with two Hard to be extant subscales additionally clustering together within the same factor (i.e., Hard to be Involved and Sociable). Although results were provided regarding the performance of the other subscales in subsequent analyses, such information needs to be treated with a degree of caution due to Factors 5 to 8 in the matrix being a poor replication of the extant structure in the current sample of adults with mental retardation. It appears that clinicians and researchers working in the field of mental retardation can use the Hard to be Involved and Sociable, Hard to be Assertive, Hard to be Supportive, Too Aggressive subscales and the full-scale Inventory of Interpersonal Prob- American Association on Mental Retardation 139

5 Table 2. Item-Factor Loading Matrix for the Inventory of Interpersonal Problems-32 (IIP-32) Items and Extant Structure Original IIP-32 scale structure (item number) Hard to be Sociable Scale Hard to socialize with other people (7) Hard to make friends (3) Hard to join in on groups (1) Hard to feel comfortable around other people (9) Hard to be Assertive Scale Hard to be assertive with another person (2) Hard to be firm when I need to be (11) Hard to be aggressive toward other people when the situation calls for it (6) Hard to disagree with other people (4) Too Aggressive Scale I lose my temper too easily (28) I argue with others too much (30) I fight with people too much (20) I get irritated or annoyed too easily (21) Too Open Scale Hard to tell personal things to other people (10) I open up to people too much (24) I tell personal things to other people too much (29) Hard to open up and tell my feelings to another person (17) Too Caring I put other people s needs before my own too much (25) Continued 140 American Association on Mental Retardation

6 Table 2. Continued Original IIP-32 scale structure (item number) Hard to attend to my own welfare when somebody else is needy (18) I am overly generous to other people (26) I am affected by another person s misery too much (32) Hard to be Supportive Hard to take instructions from people who have authority over me (16) Hard to really care about other people s problems (14) Hard to put someone else s needs before my own (15) Hard to be supportive of another person s goals in life (13) Hard to be Involved Scale Hard to make a long-term commitment to another person (5) Hard to be involved with another person without feeling trapped (19) Hard to experience a feeling of love for another person (12) Hard to show affection for another person (8) Too Dependent Scale I am too envious and jealous of other people (31) I worry too much about other people s reactions to me (27) I want people to admire me too much (22) I am too dependent on other people (23) Note. Scones in boldface are those considered to be a member of the factor American Association on Mental Retardation 141

7 Table 3. Inventory of Interpersonal Problems-32 (IIP-32) Reliability Results: Stability and Internal Consistency Scale/Subscale Test retest total sample Total sample Coefficient Community sample Clinical sample Full IIP-32.84** Hard to be Assertive.78** Hard to be Sociable.75** Hard to be Supportive.70** Hard to be Involved.69** Too Aggressive.55** Too Open.60** Too Caring.72** Too Dependent.51** **p.01. lems-32 with a degree of confidence due to the validity and reliability evidence presented. It appears that there was a difference between the Hard to be subscales and the Too Much subscales in terms of the factor analysis results, with lower replication rates evident in the Too Much subscales. The Too Much items requires self-appraisals that differ in cognitive complexity to the Hard to be items, with attendant cognitive loads possibly influencing the response patterns of adults with mild mental retardation. This issue regarding the Inventory of Interpersonal Problems-32 will require further research. However, perfect replications of factor structures are rare, and significant overlaps more common (Floyd & Widaman, 1995). Confirmatory factor analysis of the Inventory of Interpersonal Problems-32 in people with mild mental retardation represents a further appropriate research goal, before consideration is given to whole-hearted adoption of an alternative scale construction. The eight subscales significantly correlated with the Interpersonal Sensitivity scale of the Brief Symptom Inventory, indicating evidence of concurrent validity. Evidence of sound concurrent validity was further reinforced via the illustration of the fullscale Inventory of Interpersonal Problems-32 having the highest correlation with the Interpersonal Sensitivity scale out of the nine possible primary symptom scales of the Brief Symptom Inventory. The Inventory of Interpersonal Problems-32 retained internal and external criterion-related va- Table 4. Means and SDs Across Scales by Samples and t Tests Comparing Samples Scale/Subscale Total sample Mean SD Community sample Mean SD Clinical sample Mean SD t Full IIP * Hard to be Assertive Hard to be Sociable Hard to be Supportive Hard to be Involved Too Aggressive * Too Open Too Caring Too Dependent *p American Association on Mental Retardation

8 lidity. The ability of this Inventory to discriminate between the clinical and community group was illustrated by significant t tests for the full-scale and the Too Aggressive subscale. The mean scores on this Inventory for the community group raise the possibility that community participants may have been experiencing levels of interpersonal distress similar to those of the clinical group, but may have been internalizing such problems and, therefore, not coming to the attention of services. The Inventory of Interpersonal Problems-32 illustrated satisfactory test retest results, indicating stability in the test over time. It was not possible to complete test retest analysis on the clinical sample because that sample was engaged in treatment, which would in all likelihood alter interpersonal problems and, hence, invalidate this form of reliability analysis. Beail (2000), however, has previously illustrated that the Inventory of Interpersonal Problems-32 is sensitive to change when patients with mild mental retardation were in receipt of psychodynamic psychotherapy. The current results, in tandem with the Beail (2000) data, indicate that the Inventory of Interpersonal Problems- 32 is a scale with sufficient stability and sensitivity for adults with mental retardation. The internal reliability of the subscales was generally satisfactory, but an issue of specific concern is the Too Open subscale. In both clinical and community groups, the scale had poor internal reliability. Clinicians and researchers should, therefore, regard the Too Open subscale with a significant degree of caution but not discard it completely, because it contributes to the composition of the Inventory of Interpersonal Problems-32 full-scale score. An issue of methodological concern regarding validity issues for the current study is the fact that all data were self-report; therefore, the results are potentially influenced and flawed due to the extraneous effects of common method variance. Methodologically, the current study would have benefited from the employment of other sources of information on interpersonal problems against which the self-reports could have been compared and usefully triangulated (Windle, 1992). A major future methodological advance would be for researchers to triangulate the source Inventory of Interpersonal Problems-32 data between the person-referred, clinician ratings, and ratings from key significant others in the person s life, with a detailed working knowledge of core and typical means of relating. Clinicians are motivated to strike a compromise between the relative purity of focus that unitary comprehensive measures provide and the symptom breadth provided by the application of multiple measures. Cronbach and Gleser (1965) classically referred to this clinical predicament as that of a bandwidth-fidelity dilemma, which relates to the degree of complexity/scope indexed by a measure, which is placed in a dynamic compromise with fidelity (i.e., the dependability of that measure). In the clinical situation, practitioners are motivated to measure the greatest amount of patient information within the least amount of clinical time. Hence, a critical issue is the relative length of measures, the strain imposed on patients in completing the measures, and the quality of the eventual information gleaned. Cronbach (1970) stated: When several questions are of about equal importance, it is more profitable to use a brief test giving a rough answer to each one, than to use a precise test answering only one or two questions (p. 180). The Inventory of Interpersonal Problems-32 does appear to provide a compromise to the bandwidth fidelity dilemma in the assessment of interpersonal problems in people with mild mental retardation, because although the measure is short, it accesses with a degree of reliability and validity relatively distinct areas and aspects of interpersonal dysfunction. References Aiken, A. R. (1996). Rating scales and checklists: Evaluating behavior, personality and attitude. New York: Wiley. Barkham, M., Hardy, G. E., & Startup, M. (1996). The Inventory of Problems-32: A short version of the Inventory of Interpersonal Problems. British Journal of Clinical Psychology, 35, Beail, N. (2000). An evaluation of outpatient psychodynamic psychotherapy for adults with intellectual disabilities. Journal of Intellectual Disability Research, 44, 204. Beail, N. (2003). What works for people with mental retardation? Critical commentary on cognitive behavioral and psychodynamic psychotherapy research. Mental Retardation, 41, Beail, N., & Warden, S. (1996). Evaluation of a psychodynamic psychotherapy service for adults with intellectual disabilities: Rationale, design and preliminary outcome data. Journal American Association on Mental Retardation 143

9 of Applied Research in Intellectual Disabilities, 9, Bramston, P., & Fogarty, G. (2000). The assessment of emotional distress experienced by people with an intellectual disability: A study of different methodologies. Research in Developmental Disabilities, 21, Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, Cronbach, L. J. (1970). Essentials of psychological testing (3rd ed.). New York: Harper. Cronbach, L. J., & Gleser, G. C. (1965). Psychological tests and personal decisions (2nd ed.). Urbana, IL: University of Illinois Press. Derogatis, L. R. (1993). Brief Symptom Inventory: Administration scoring and procedures manual (3rd ed.). Minneapolis: National Computer Systems. Everett, J. E. (1983). Factor compatibility as a means of determining the number of factors and their rotations. Multivariate Behavioral Research, 2, Finlay, W. M. L., & Lyons, E. (2001). Methodological issues in interviewing and using selfreport questionnaires with people with mental retardation. Psychological Assessment, 13, Floyd, F., & Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7, Guttman, L. (1954). Some necessary and sufficient conditions for common factor analysis. Psychometrika, 19, Harman, H. H. (1967). Modern factor analysis (2nd ed.). Chicago: Chicago University Press. Horowitz, L. M., Rosenberg, S. E., Baer, B. A., Ureno, G., & Villasonor, V. S. (1988). Inventory of Interpersonal Problems: Psychometric properties and clinical applications. Journal of Consulting and Clinical Psychology, 56, Kellett, S. C., Beail, N., Newman, D. W., & Mosley, E. (1999). Indexing psychological distress in people with an intellectual disability: Use of the Symptom Checklist-90-R. Journal of Applied Research in Intellectual Disabilities, 12, Kellett, S. C., Beail, N., Newman, D. W., & Frankish, P. (2003). Utility of the Brief Symptom Inventory (BSI) in the assessment of psychological distress. Journal of Applied Research in Intellectual Disabilities, 16, Kellett, S. Beail, N, Newman, D. W., & Hawes, A. (2004). Factor structure of the Brief Symptom Inventory: Intellectual disabilities evidence. Clinical Psychology and Psychotherapy, 11, Mindham, J., & Espie, C. A. (2003). Glasgow Anxiety Scale for People With an Intellectual Disability (GAS-ID): Development and psychometric properties of a new measure for use with people with mild intellectual disability. Journal of Intellectual Disability Research, 47, Nunnally, J. (1970). Introduction to psychological measurement. New York: McGraw-Hill. Powell, R. (2003). Psychometric properties of the Beck Depression Inventory and the Zung Self Rating Depression Scale in Adults With Mental Retardation. Mental Retardation, 41, Prout, H. T., & Nowack-Drabick, K. M. (2003). Psychotherapy with persons who have mental retardation: An evaluation of effectiveness. American Journal on Mental Retardation, 108, Roth, A., & Fonagy, P. (1996). What works for whom? New York: Guildford Press. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations; uses in assessing rater reliability. Psychological Bulletin, 86, Tabachnick, B. G., & Fidell, L. S. (1989). Using multivariate statistics (2nd ed.). New York: Harper Collins. Windle, M. (1992). Revised Dimensions of Temperament Survey (DOTS-R): Simultaneous group confirmatory factor analysis for adolescent gender groups. Psychological Assessment, 4, Received 11/04/03, accepted 9/23/04. Editor-in-charge: William E. MacLean, Jr. The second author is also affiliated with the University of Sheffield. Requests for reprints should be sent to Stephen Kellett. Psychological Healthcare, Keresforth Centre, Barnsley S70 6RS, United Kingdom. stephen.kellett@barnsleypct. nhs.uk 144 American Association on Mental Retardation