The Rorschach Comprehensive System. and Objectivity: Issues. Of Validity, Reliability, and Bias. Ellen B. Aquino. Daemen College

Transcription

1 Rorschach Objectivity Issues 1 Running head: RORSCHACH OBJECTIVITY ISSUES The Rorschach Comprehensive System and Objectivity: Issues Of Validity, Reliability, and Bias Ellen B. Aquino Daemen College Assessment Methods in Education April 30, 2003 (Note. Used with permission. Copyright by Ellen B. Aquino, 2003)

2 Rorschach Objectivity Issues 2 Abstract The publication of Exner s (1996) The Rorschach: A Comprehensive System appeared to resolve all historical debate regarding the issue of standardizing the Rorscach technique of projective testing. However, recent study in this genre raises serious questions regarding objective issues surrounding Exner s Rorschach method. Specifically, concerns regarding reliability, validity, and the possible existence of cultural bias plague psychiatrists. However, a review of current research demonstrates that concerns of cultural bias have been allayed, despite continued problems plaguing reliability and validity.

3 Rorschach Objectivity Issues 3 The Rorschach Comprehensive System and Objectivity: Issues Of Validity, Reliability, and Bias Possessing an extensive and varied history, Rorschach testing has been in existence for approximately 85 years. This procedure experienced a figurative rebirth in the 1970s, when John Exner (1996) published his now famous The Rorschach: A Comprehensive System (herein TRACS). However, despite such historical foundations, this oftencontroversial test has been the focus of recent research and debate. Specifically, contemporary professionals in the psychological field have raised prime concerns regarding the objectivity of TRACS, with unambiguous worries regarding such issues as validity, reliability, and bias. These concerns raise questions surrounding the supposed standardization and normative procedures that Exner executed prior to publication of TRACS. The Rorschach method of testing first originated in 1922 by a Swiss psychiatrist named Hermann Rorschach; he had been experimenting with inkblot testing for many years prior to this date (Ford, 1975). Rorschach formed said inkblots by folding over a piece of paper into the center of which ink had been dropped (Rorschach Inkblot Test,

4 Rorschach Objectivity Issues , para. 1). Rorschach s testing method is classified within the psychological community as a projective personality test that requires people to interpret ambiguous inkblots (Nairne, 2000, p. 471). The projective personality measure is one of two major classifications for tools used to assess personality, the other being a self-report inventory; an example of the latter category is the Minnesota Multiphasic Personality Index (MMPI). A chief distinction between the two categories involves the freedom of responses possessed by the subject projective tests allow you to respond freely, in a manner that is less restrictive than the requirements imposed by self-report inventories (Nairne, 2000, p. 471). When developing the initial testing method over a span of four years, Hermann Rorschach developed probable uses for the new testing method. The vast importance of these in evident in their mere inclusion in Hock s (1999) Forty Studies That Changed Psychology: Rorschach believed that his projective technique could serve two main purposes. One was that it could be used as a research tool to reveal unconscious aspects of personality. The other purpose, claimed somewhat later by Rorschach, was that the test could be used to

5 Rorschach Objectivity Issues 5 diagnose various types of psychopathology. (p. 239) Both Rorschach and subsequent researchers developed systems to utilize when coding the inkblot tests; such standardized systems include those of Rorschach, Klopfer, Beck, Rappaport, and Piotrowski (Ritzler, 2001). However, it is essential to note that, while the Rorschach test was popular with clinicians throughout the 1940s and 1950s, it fell into disfavor shortly thereafter. Many clinicians began criticizing it as subjective...in nature (Matthews, 1999, para. 1). Exner and contemporaries perceived major concerns with each of the previously mentioned coding systems. Specifically, debate in the 1960s was heated; critics identified problems intricately related to validity and bias while defenders asserted that criticism of the test was naïve (Wood, Nezworski, & Stejskal, 1996). According to Ritzler (2001), to address these concerns, Exner: Systematically surveyed the previously mentioned systems and selected coding variables...he used the resulting data to develop a composite coding and interpretation system that became known as the Comprehensive System, which has become the most Frequently used coding and interpretive method of Rorschach assessment. (p. 238)

6 Rorschach Objectivity Issues 6 Through this publication, Exner has almost single handedly rescued a drowning beast and breathed life back into it (Matthews, 2003, para. 3), saving the test from the total disrepute into which it was rapidly sinking (Campbell, 1999, para. 3). TRACS would subsequently be revised through a total of three volumes. From a critical standpoint, however, it remains imperative that TRACS, though often attributed with satisfying both sides in the Rorschach controversy is hardly impervious to scrutiny (Wood, et al., 1996, p. 3). Rather, it appears necessary that the normative and standardization procedures executed by Exner prior to the publication of TRACS be considered, in conjunction with current research on related issues. Reliability, validity, and bias are all logically connected with the notion of standardized testing and deserve adequate consideration (Popham, 2002). In TRACS, Exner himself concedes that the notion of normalcy relies primarily on context. He also asserts that it is essential that the interpreter of data have some baseline or guidelines to help him or her judge normal from abnormal features, and put these features in the proper perspective (Exner and Weiner, 1982, p. 42). Subsequently, Exner designed a process for obtaining

7 Rorschach Objectivity Issues 7 normative data in which he selected a large representative sample of children, with approximately equal gender quantities, and a stratification on the basis of socioeconomic status (SES) and geography, according to age level (Exner and Weiner, 1982). These descriptions refer to the specific process used in the creation of TRACS Volume Three, which focused solely on the assessment of children and young adults. It is from the Rorschach responses of this representative sample that norms were drawn. The differences that manifest themselves among and across the groups are not so frequent or substantial to argue for separate normative data by sex, by socioeconomic level, or by region (Exner and Weiner, 1982, p. 42). Within the psychological community, little criticism has been directed at this norming process; critical focus has rather emphasized the supposed standardized scoring methods outline in the text. Although standardization can (and will) be addressed in regard to the presence or absence of culture bias, it also focuses largely on the presentation and coding/scoring methods used with TRACS, as Ritzler (2001) contends: Proper application of the Rorschach Comprehensive System requires a standard administration followed by a complete coding of all responses and subsequent

8 Rorschach Objectivity Issues 8 interpretation using a systematic procedure ensuring that all who use it cover all variables in the system...in an invariable step-by-step fashion tailored for each individual protocol. (p. 239) Each of the ten inkblot plates must be presented in order twice, first during a Free Association Period and then in Inquiry as the therapist questions without leading (Psychological Counseling, 2002). All words spoken by the client must be recorded, and all such information must be coded. Responses are coded under such categories as pair (reflects two of an object) or reflection (refers to a mirror image); such categories are consequently considered indications of personality characteristics. Specifically, pairs and reflections indicate self-esteem, self-focus, and narcissism on the Egocentricity Index (Wood et al., 1996). There are nine codes total in TRACS, a key code being popular (most likely responses for a given plate). A single response from a client can be assigned multiple codes a simple response might receive five, a complex responses as many as 15 resulting in Rorschach tests of 23 responses involving several hundred code combinations (Guarnaccia, Dill, Sabatino, and Southwick, 2001). The implementation of such complex coding systems was intended to improve the clinical natural of the Rorschach

9 Rorschach Objectivity Issues 9 testing method. However, it raises serious questions amongst professionals, both in terms of the codes and the method. Some resent the notion of a created popular interpretation of a supposedly ambiguous inkblot; the experts invented standardized interpretations of interpretations (Carroll, 2002, para. 5). In reference to the methods, Guarnaccia et al. (2001) compared the scoring accuracy of (recently trained in Rorschach methods) graduate students and clinical professionals. Each group scored Rorschach responses drawn from clinical and normal protocols. Although demonstrating high-rates of interrater reliability, both groups performed below acceptable levels (80%) in regards to inter-scoring session accuracy (Guarnaccia et al., 2001). It is perhaps Exner (1996) himself, in his response to the work of Woods et al., who provides the soundest resolution to such dispute: I have expressed my hope that young researchers will take up the challenge to refine the system and, in doing so, contribute more information about the test and, more important, about people and ways to aid them in their times of quandary. (p. 13) Ultimately, further research is required to settle all disputes regarding the validity and reliability of TRACS.

10 Rorschach Objectivity Issues 10 With regard to the presence of absence of cultural bias, the format of Rorschach testing is often cited as the prime reason for the avoidance of bias; the test stimuli (inkblots) are sufficiently ambiguous to eliminate most cultural bias (Ritzler, 2001, p. 238). Similarly, it is unlikely that the inkblots resemble specific items from any particular culture, as popular (normative) answers are often easily transferred from culture to culture. Such responses include animal, human head, butterfly, and similar culturally common objects (Ritzler). Also, the previously discussed standardized method of administration is intended to assist in the elimination of cultural bias. However, because the test structure itself seems relatively free-of-bias, professionals have raised more complex questions. Specifically, such queries have centered on the possibility of varied test performance by minorities as a result of shared experience (differing from that of the majority) or language/idiomatic variances (Meyer, 2002; Presley, Smith, Hilsenroth and Exner, 2001). In the Presley et al. (2001) study, groups of African American and White American [sic] were matched for several variables, including education, age, and SES; Rorschach test scores were then compared. Of 23 Rorschach variables studied, only three were statistically different between the two

11 Rorschach Objectivity Issues 11 ethnicities. This striking similarity suggests the absence of cultural bias within TRACS. Meyer s (2002) work comparing the Rorschach results of European Americans and non-european Americans supports this notion, finding that once matched with regard to education, gender, marital status, and age, any seemingly significant differences (i.e. in the number of responses) were virtually eliminated. Meyer asserts that, in the case of TRACS, the absence of ethnic bias should be the default expectation (para. 9). While Presley et al. (2001) concede to this notion, it is significant to note that with regards to cooperative movement (COP a code used in scoring) responses on the test, White Americans reported a mean of one COP while African Americans reported a mean of approximately one. When this data is viewed in connection with patient progress, patients with increased COP responses exhibited positive progress. Of patients with two or more COP responses, 81% had favorable progress at the time of discharge. This finding suggests that African Americans are less likely to perceive or anticipate positive interactions among others as routine events and may reflect the results of an anticipation of racial discrimination (Presley et al., para. 2). While not altering the validity between ethnic groups, and thereby

12 Rorschach Objectivity Issues 12 not signaling the presence of cultural bias, this notion begins to form a response pattern specific to a minority groups. Though the presence of cultural bias appears disproved, the notion of specific response patterns raises a new area of interest for researchers. Therefore, it appears that the objectivity of TRACS is not seriously considered by researchers, concentrating on issues of scoring accuracy and reliability; cultural bias as a concern seems to be eradicated. Questions obviously remain regarding the utilization of TRACS in clinical settings is it feasible to utilize a diagnostic technique that is consistently the focus of such scrutiny? However, in reality, most clinicians find their uses of TRACS as a sort of compromise between these two extremes, wherein it is not used as a formal test, but rather as a means of increasing a therapist s understanding of individual clients (Hock, 1999, p. 244). This utilization appears as a means of temporary resolution in the ongoing Rorschach debate until further definitive research into areas including validity and reliability can be conduct. Ideally, additional research will function as that in the area of cultural bias did, allaying virtual all clinical concerns.

13 Rorschach Objectivity Issues 13 References Campbell, T. W. (1999). Issues in forensic psychology Rorschach technique. Retrieved April 28, 2003 from Carroll, R. T. (2002). Rorschach inkblot test. Retrieved April 29, 2003 from inkblot.html Exner, J. E., Jr. (1996). A comment on The Comprehensive System for the Rorschach: A critical examination. Psychological Science, 7(1), Exner, J. E., Jr. & Weiner, I. B. (1982). The Rorschach A Comprehensive System: Vol. 3 Assessment of children and adults. New York: John Wiley and Sons. Ford, M. (1975). The application of the Rorschach test to young children. Westport, CT: Greenwood Press. Guarnaccia, V., Dill, C. A., Sabatino, S., & Southwick, S. (2001). Scoring accuracy using the Comprehensive System for the Rorschach. Journal of Personality Assessment, 77(3), Retrieved April 18, 2003 From EBSCOhost ( ). Hock, H. R. (1999). Forty studies that changed psychology: Explorations into the history of psychological reason. Upper Saddle River, NJ: Prentice Hall.

14 Rorschach Objectivity Issues 14 Matthews, M. (1999). Rorschach inkblot test. Retrieved April 28, 2003 from Meyer, G. J. (2002). Exploring possible ethnic differences and bias in the Rorschach Comprehensive System. Journal of Personality Assessment, 78(1), Retrieved April 18, 2003 from EBSCOhost ( ). Nairne, J. S. (2000). Psychology: The adaptive mind. Belmont, CA: Wadsworth. Popham, W. J. (2002). Classroom assessment: What teachers need to know (3rd ed.). Boston: Allyn and Bacon. Presley, G., Smith, C., Hilsenroth, M., and Exner, J. (2001). Clinical utility of the Rorschach with African Americans. Journal of Personality Assessment, 77(3), Retrieved April 18, 2003 from EBSCOhost ( ). Psychological Counseling. (2002). Retrieved April 29, 2003 from Ritzler, B. (2001). Multicultural usage of the Rorschach. In L. A. Suzuki, J. G. Ponterotto, & P. J. Meller (Eds.), Handbook of multicultural assessment: clinical, psychological, and educational application. (pp ). San Francisco: Jossey-Bass.

15 Rorschach Objectivity Issues 15 Rorschach Inkblot Test. (2003). Retrieved April 28, 2003 from Wood, J. M., Nezworski, M. T., & Stejskal, W. J. (1996). The Comprehensive System for the Rorschach: A critical examination. Psychological Science, 7(1), 3-10.