J. Indian Assoc. Child Adolesc. Ment. Health 2008; 4(4): Original Article

Transcription

1 88 J. Indian Assoc. Child Adolesc. Ment. Health 2008; 4(4):88-94 Original Article Inter- and Intra-rater Reliability of the Draw-A-Person Test among Indian Children and Adolescents Garima Srivastava, MA, Anubha Dhal, PhD, Geetanjali Chugh, PhD, Manju Mehta PhD, Rajesh Sagar, MD, V Sreenivas, PhD Address for correspondence: Ms. Garima Srivastava, PhD Scholar, Department of Psychiatry, All India Institute of Medical Sciences, New Delhi , srivastavagarima9@gmail.com ABSTRACT Objectives: To establish the inter- and intra-rater reliability of the Draw-A-Person Test among Indian children and adolescents. Method: 90 child and adolescent outpatients aged 7-15 years and diagnosed with anxiety, depression, somatoform disorders, or attention deficit hyperactivity disorder were administered the Draw-A-Person Test. 50 randomly selected Draw-A-Person Test (DAPT) protocols were scored independently by three trained psychologists. The inter- and intra-rater reliability was established using the Kappa test for assessing agreeability (% and frequency) and McNemar analysis to assess discordance. Results: Perfect intra- and inter-rater agreement ( ) was obtained for 130 variables. A substantial ( ) intra-rater agreement was noted for 5 variables and inter-rater agreement ( ) was noted for 2 variables; and a moderate inter- and intra-rater agreement ( ) was noted for one variable. was not significant for any variable. Conclusions: The Draw-A-Person Test was found to have a high inter- and intra-rater reliability when administered on Indian children with psychiatric disorders. Key Words: Draw-A-Person Test (DAPT), Inter-rater reliability, Intra-rater reliability, INTRODUCTION The Draw-A-Person Test (DAPT) was developed by Machover in 1949 and uses figure drawings in a projective way, focusing on how the drawings reflects the anxieties, impulses, selfesteem and personality of the test taker. 1 The DAPT like most projective techniques has been widely used in clinical and forensic setups due to its capacity to circumvent conscious defenses of respondents and garner information regarding the unconscious psychological aspects like conflicts and impulses, but its scientific status remains highly controversial. It is based on the projective hypothesis that states that persons project aspects of their personalities in the process of disambiguating unstructured stimuli. 2 Literature is replete with human figure drawings used as a mode of psychological assessment for cognitive and developmental functioning of children, particularly for identifying indicators for children with psychiatric disorders like Aspergers Syndrome and Autism. 3 However, there is sparse evidence for reliability of clinicians interpretation of the indicators or indices though many interpretations do not follow directly from figure drawing scores. 2,4 Reliability refers to the consistency of ratings or to the ability of two or more reviewers to reach the same conclusions about a specific case. Intra-rater reliability is the consistency of

2 successive measurements by an individual rater and inter-rater reliability is used to measure consistency of agreement between two independent raters. 5 Though some researchers have used DAPT for assessment of internalizing and externalizing disorders in Indian children; 6 there has been little attempt to validate the test for use in India. Hence, the aim of the present study was to establish the inter- and intra-rater reliability of the DAPT among Indian children and adolescents with psychiatric disorders. METHOD Ninety 7-15 year-old children and adolescent outpatients attending a Child and Adolescent Guidance Clinic and diagnosed with anxiety, depression, somatoform, attention deficit hyperactivity disorder or conduct disorder (without comorbidity) as per DSM-IV TR criteria were included in the study. Prior ethical clearance from the Institute Ethics Committee was obtained and written informed consent was taken from parents/guardians. The participants were administered the socio-demographic Performa, to obtain information about sociodemographic variables like name, age, sex, religion and family type; and the DAPT. The investigator was blind to the diagnosis at the point of referral and three trained psychologists were blind to identifying information (of subjects) while scoring and interpreting the protocol. Fifty protocols were randomly selected for assessing inter- and intra-rater reliability and were rated independently by three psychologists. The administration and interpretation of DAPT was done using The Draw-A-Person Catalogue for Interpretative Analysis. 7 A structured proforma for rating 130 commonly rated variables was developed for categorization and coding of qualitative signs observed in the drawings of children based on the catalogue, e.g. Head normal: Present-0, Absent-1, Buttons: Present-0, Absent-1. The coding/scores assigned for each variable by the raters were used to establish inter-rater reliability. To establish intra rater reliability, protocols were rated twice by the same rater after a span of 1 month. Inter-rater and intra-rater reliabilities for key judgments were calculated using the kappa statistic (κ) using STATA for Windows Version 9.0. Three-way kappa was used to evaluate reliability between all three raters and two-way kappa analysis performed for evaluation of rater pairs. Because the marginal totals for some outcomes for some pairs of raters were very different, the maximum possible value of kappa was also calculated. Kappa max (max κ) was calculated using the equation: 1 (minimum disagreement/expected disagreement) The Mc Nemar test was used to estimate the discordance at α=0.05 level of significance. RESULTS The total sample consisted of 90 children and adolescents with mean age of 12±2.3years. Males constituted 59% and females 41% of the sample. Majority (51.1%) of children were enrolled in private schools with 24.4% in primary, 44.4% secondary and 31.1% in higher education. About 76% of the children belonged to Hindu household with 80% residing in urban areas and 41% belonging to upper and 22% belonging to lower socio economic strata. About 61% of children were living in a joint family setup constituting between four to eight household members. Nearly 39% of children had one sibling and 27% had two siblings and 27% had no sibling. 89

3 Table 1: Inter-rater and Intra-rater reliability (agreement) on DAPT Projective Factors Intra-rater Agreement Inter-rater Agreement Sex Drawn 0.95 ( ) ( ) 0.99 Age Drawn 1.00 ( ) ( ) 0.99 Groundline Treatment 0.91 ( ) ( ) 0.99 Size of the Figure 0.96 ( ) ( ) 0.99 Placement 0.96 ( ) ( ) 0.99 Transparency 1.00 ( ) ( ) 0.99 Head Size 0.92 ( ) ( ) 0.99 Reinforced 0.91 ( ) ( ) 0.99 Irregular 1.00 ( ) ( ) 0.99 Drawn backwards 1.00 ( ) ( ) 0.99 Facial Features Dim 1.00 ( ) ( ) 0.99 Hair Normal 0.96 ( ) ( ) 0.99 Shaded 0.95 ( ) ( ) 0.99 Long but un-shaded 0.78 ( ) ( ) 0.99 Omitted 0.88 ( ) ( ) 0.99 Disarray 1.00 (72-100) ( ) 0.99 Emphasis 0.64 ( ) ( ) 0.99 Ear Normal 100 (72-100) ( ) 0.99 Omitted 0.92 ( ) ( ) 0.5 Large 1.00 ( ) ( ) 0.99 Emphasis 0.94 ( ) ( ) 0.99 Eyes Large 1.00 ( ) ( ) 0.99 Small 0.95 ( ) ( ) 0.99 Dot 0.90 ( ) ( ) 0.99 Empty 0.49 ( ) ( ) 0.99 Emphasis 1.00 ( ) ( ) 0.99 Nose Normal 0.96 ( ) ( ) 0.99 Emphasis 1.00 ( ) ( ) 0.99 Mouth Normal 0.93 ( ) ( ) 0.99 Emphasis 1.00 ( ) ( ) 0.99 Smiling 0.94 ( ) ( ) 0.99 Open 1.00 ( ) ( ) 0.99 Object in mouth 1.00 ( ) ( ) 0.99 Slash line 0.95 ( ) (67-100) 0.99 Teeth 1.00 ( ) ( ) 0.99 Upturned 1.00 ( ) ( ) 0.99 Concave 1.00 ( ) ( ) 0.99 Chin Normal 0.95 ( ) ( ) 0.99 Emphasis 0.90( ) ( )

4 Table 1 (Cont.): Inter-rater and Intra-rater reliability (agreement) on DAPT Projective Factors Intra-rater Agreement Inter-rater Agreement Neck Normal 0.96 ( ) ( ) 0.99 Short and Thick 1.00 ( ) ( ) 0.99 Thick and Long 0.85 ( ) ( ) 0.99 Long and Thin 1.00 ( ) ( ) 0.99 Shaded 0.85( ) ( ) 0.99 Trunk Large 0.66 ( ) ( ) 0.99 Thin and Narrow 0.95 ( ) ( ) 0.99 Round 1.00 ( ) ( ) 0.99 Large Breast 1.00 ( ) ( ) 0.99 Waist Normal 0.96 ( ) ( ) 0.99 Heavy 0.85 ( ) ( ) 0.99 Tightened 1.00 ( ) ( ) 0.99 Emphasis 1.00 ( ) ( ) 0.99 Shoulder Erasure 1.00 ( ) ( ) 0.99 Large 0.89 ( ) ( ) 0.99 Tiny 0.96 ( ) ( ) 0.99 Unequal 1.00 ( ) ( ) 0.99 Emphasis by female 1.00 ( ) ( ) 0.99 Emphasis by male 0.66 ( ) ( ) 0.99 Arms Broad 1.00 ( ) ( ) 0.99 At the back 0.88 ( ) ( ) 0.99 On the waist 1.00 ( ) ( ) 0.99 Outstretched 0.95 ( ) ( ) 0.99 Folded 1.00 ( ) ( ) 0.99 Reinforced 1.00 ( ) ( ) 0.99 Unattached 0.66 ( ) ( ) 0.99 Hanging 1.00 ( ) ( ) 0.99 Hands Normal 0.92 ( ) ( ) 0.99 Behind 1.00 ( ) ( ) 0.99 Mitten 1.00 ( ) ( ) 0.99 Shaded 1.00 ( ) ( ) 0.99 Fingers Clenched 1.00 ( ) ( ) 0.99 Fewer than five 1.00 ( ) ( ) 0.99 More than five 1.00 ( ) ( ) 0.99 Grape like 1.00 ( ) ( ) 0.99 Thick 1.00 ( ) ( )

5 Table 1 (Cont.): Inter-rater and Intra-rater reliability (agreement) on DAPT Projective Factors Intra-rater Agreement Inter-rater Agreement Scribbled 1.00 ( ) ( ) 0.99 Legs Chopped 1.00 ( ) ( ) 0.99 Crossed 1.00 ( ) ( ) 0.99 Short 1.00 ( ) ( ) 0.99 Pressed together 1.00 ( ) ( ) 0.99 Reinforced 1.00 ( ) ( ) 0.99 Short 1.00 ( ) ( ) 0.99 Feet Bare 1.00 ( ) ( ) 0.99 Emphasis 0.95 ( ) ( ) 0.99 Large 1.00 ( ) ( ) 0.99 Over-detailing 1.00 ( ) ( ) 0.99 Pointed 1.00 ( ) ( ) 0.99 Opposite direction 1.00 ( ) ( ) 0.99 Same direction 1.00 ( ) ( ) 0.99 Small 0.92( ) ( ) 0.99 Toes Pointed 1.00 ( ) ( ) 0.99 Posture Normal 0.92 ( ) ( ) 0.99 Stiff 1.00 ( ) ( ) 0.99 Action figure 1.00 ( ) ( ) 0.99 Front view 0.95 ( ) ( ) 0.99 Profile view 1.00 ( ) ( ) 0.99 Stance Wide 0.92 ( ) ( ) 0.99 Clothing Over-detailing 1.00 ( ) ( ) 0.99 Accessories Button 1.00 ( ) ( ) 0.99 Pocket 1.00 ( ) ( ) 0.99 Tie 1.00 ( ) ( ) 0.99 Shoe 1.00 ( ) ( ) 0.99 Belt 1.00 ( ) ( ) 0.99 Jewelry 1.00 ( ) ( ) 0.99 Cap 1.00 ( ) ( ) 0.99 Earrings 0.92 ( ) ( ) 0.99 Strokes Erasure 0.95 ( ) ( ) 0.99 Shading 0.90 ( ) ( ) 0.99 Pencil pressure 1.00 ( ) ( ) 0.99 Discontinuous strokes 1.00 ( ) ( ) 0.99 Light strokes 1.00 ( ) ( )

6 The inter-rater and intra-rater discordance was very low (0.99, not significant) for all variables, except empty eyes (0.5) on inter-rater and omitted ears (0.5) for intra-rater agreement. Inter-rater and intra-rater agreement was high (κ= ) and highly significant for most variables (Table 1). Intra-rater agreement was substantial for long and un-shaded hair, hair emphasis, large trunk and shoulder emphasis variables; whereas the three raters were in substantial agreement for variables pertaining to hair emphasis and unattached arms (κ= ). The intra and inter rater agreement was moderate for empty eyes (κ= ). DISCUSSION The evidence regarding psychometric properties of the Draw-A-Person Test has been poorly documented. The intra and inter rater agreement was high for all projective variables of social and emotional development indicated by the Draw-A-Person Test. This is in contrast with previous literature, which suggests that the reliability of judgments of children's social and emotional development and personality indicators was insufficient. 8 This may be explained based on the modest training imparted to raters in previous research studies in comparison to the present study where the raters were adequately trained in the scoring and interpretation of the Draw-A-Person Test. Adequate training of raters helps to impose some level of objectivity onto the process and hence increases the consensus in ratings assigned on different indices. The inter- and intra-rater agreement was almost perfect for one hundred and thirty variables. Inter-rater agreement was also substantial for three projective variables i.e. long and un-shaded hair, large trunk and shoulder emphasis and moderate for projective variables pertaining to hair emphasis and empty eyes. Relatively lower agreement on these variables can be accounted for by the difficulties in interpretation caused by differences in drawing abilities of children of different age or due to personal and situational variability in raters as well as poor operational definitions of the terms mentioned in the interpretation manual. This is in line with the previous literature that suggests that the ratings assigned may depend on the rater s subjective interpretation of the criteria. For example, hair emphasis may be used to denote extra pencil pressure while drawing hair or may also be used to denote long or spread out hair. Similarly, empty eyes might have been used for rating drawings in which eyelids were drawn closed and pupils might not have been visible which the raters may have confused with closed eyes or drawing a faint dot in place of eyes. The present study contributes to the existing literature base on psychometric properties of Draw-A-Person test by helping establish it s inter and intra rater reliability in the Indian context. It also proposes a structured and objective way to evaluate Draw-A-Person test, thereby addressing some criticism which limits the usage of this projective test in clinical setting because of the subjectivity in scoring. However, limitations such as small sample size and restriction of sample to children and adolescents with psychiatric disorders may limit the generalizability of the findings. Further studies can include item analysis to understand and describe the relationship of individual variables used for scoring the Draw-A-Person Test, which would also add to the internal structure validity of the test. REFERENCES Machover K. Personality projection in the drawing of the human figure. Springfield,III.:Charles C. Thomas; Lilienfeld SO, Wood JM, Garb HN. The scientific status of projective techniques. Psychological Science in the Public Interest 2000; 1: Lim HK, Slaughter V. Brief report: human figure drawings by children with Asperger's syndrome. Journal of Autism Developmental Disorder 2008; 38:

7 Thomas GV, Jolley RP. Drawing conclusions: A re-examination of empirical and conceptual bases for psychological evaluations of children from their drawings. British Journal of Clinical Psychology 1998; 37: Dawson B, Trapp R. Basic and clinical biostatistics (4 th Ed.). New York: McGraw-Hill; Kohli A, Kaur M, Malhotra R. Draw A Person Test. New Delhi: Prasad Psycho Corporation; Urban WH. The Draw-A-Person Catalogue for interpretative analysis: Western psychological services; Rae G, Hyland P. Generalizability and classical test theory analyses of Koppitz's Scoring System for human figure drawings. British Journal of Educational Psychology 2001; 71: Ms. Garima Srivastava, PhD Scholar, Ms. Anubha Dhal, PhD Scholar, Dr. Geetanjali Chugh, Research Associate, Dr. Manju Mehta, Professor, and Dr. Rajesh Sagar, Additional Professor, Department of Psychiatry; and Dr. V. Sreenivas, Additional Professor, Department of Biostatistics, All India Institute of Medical Sciences, New Delhi Author s contributions: Garima Srivastava was involved in acquisition of the data, analysis and interpretation, drafting and revising the manuscript and has given final approval of the version to be published. Prof. Manju Mehta and Dr Rajesh Sagar were involved in the conception and design of the study and have given the final approval of the of the version to be published. Ms. Anubha Dhal and Dr. Geetanjali Chugh were involved in analysis and interpretation of the findings, drafting and revising the manuscript. Dr. V. Sreenivas was involved in the design of the study, analysis and interpretation. Ms. Garima Srivastava will serve as the guarantor for the study. 94