The Efficiency of the Denver Developmental Screening Test with Rural Disadvantaged Preschool Children 1

Transcription

1 Joural of Pediatric Psychology, Vol. 8, No. 3, 1983 The Efficiecy of the Dever Developmetal Screeig Test with Rural Disadvataged Preschool Childre 1 Deis C. Harper 2 ad David P. Wacker Departmet of Pediatrics, College of Medicie, Uiversity of Iowa The purpose of the study was to ivestigate the efficiecy of the Dever Developmetal Screeig Test (DDST) o a rural disadvataged preschool populatio. Several study samples were selected from a large group (1,061) of childre who were evaluated i a prospective logitudial fashio over 3 years. The study presets descriptive outcome statistics o the (1,018) childre who completed the DDST, examies the relatioship betwee screeig outcome o the DDST ad idividually admiistered itellectual measures (SB-LM or WPPS1) o a sample of 555 childre, ad evaluates the test-retest reliability of the DDST o a selected sample of 106 subjects. KEY WORDS: Dever Developmetal Screeig Test; preschool screeig; rural childre. The purpose of this study was to ivestigate the efficiecy of a widely used preschool screeig istrumet, the Dever Developmetal Screeig Test (DDST), o a rural disadvataged preschool populatio. To accomplish this goal, several study samples were selected from a larger group (1,061) of childre who were evaluated i a prospective logitudial fashio. The study provides specific outcome ad correlatioal data o a rural preschool populatio utilizig the DDST, which has ot bee described previously i 'Support for this study was made available through Materal ad Child Health grat o. MCR Portios of this study were preseted at the America Psychological Associatio meetig i Los Ageles, Califoria, 'All correspodece should be set to D. C. Harper, Departmet of Pediatrics, Office 341, Uiversity Hospital School, Iowa City, Iowa l4«-g693/83/090CM)273J03.00/l <Q 1983 Society of Ptdutric Psychdory

2 274 Harper ad Wacker the literature. The ivestigatio presets descriptive outcome statistics from evaluatig 1,018 childre with the DDST, examies the relatioship betwee screeig outcome of the DDST ad idividually admiistered itellectual measures o a sample of 555 childre, ad fially, reports testretest correlatios o the DDST for a selected group of 106 preschool-age childre. METHOD Subjects The subjects cosisted of preschool childre (x = 3.9 years) who came volutarily to commuity health screeig cliics throughout a predomiatly rural state from commuities with populatios ragig from 2,500 to 5,000 residets. All families icluded i the study were receivig federal ecoomic assistace ad were predomiatly from the lower socioecoomic level, i.e., Class IV (Holligshead, 1957, Note 1). Approximately 4 7o were miority childre, e.g., Hispaic ad black. The total sample was equally divided betwee males ( = 537) ad females ( = 524), as show i Table I. The majority of the childre resided i sigle-paret homes. The materal mea age at the iitial examiatio was 25.6 years ad 49% of the mothers had completed high school. Istrumets I additio to the DDST, the Staford-Biet, L-M (SB-LM) 3 or the Wechsler Primary ad Preschool Itelligece Scale (WPPSI) were give to selected childre. However, ot all yougsters were admiistered itellectual evaluatios because of time costraits. A stratified radom selectio procedure was used to select 555 childre o the basis of predetermied sex ad age groupigs. Admiistratio of itellectual evaluatios (SB-LM or WPPSI) was o a radom basis. Developmetal ad itelligece tests were admiistered by differet examiers i a couterbalaced fashio. All 'The 1960 editio of the Staford-Biet, L-M was used i this ivestigatio. Calculatio of metal age ad itelligece quotiets was based o the 1960 ormative data. The 1960 stadards were selected over the 1972 orms because the traditioal relatioships betwee metal age ad chroological age o loger held i may cases. Thompso (1977) oted sigificatly lower scores whe usig the 1972 orms versus 1960 criteria ad also foud variable discrepacies i IQ scores depedig o itelligece level, age, ad future fuctioal status.

3 Efficiecy of the DDST 275 Table I. Age Distributio of Childre at Iitial Screeig Examiatios Age (moths) ^ Total (" = 1,061) Vo Males % Females % assessmets were completed by traied paraprofessioals uder the supervisio of a psychologist. Procedure Psychometric examiatios were admiistered as part of cliic health evaluatios. The criterio tests (SB-LM or WPPSI) ad the DDST were admiistered by differet examiers o the same day, with either examier aware of the results. Repeat admiistratios of the DDST for test-retest aalysis were completed with a average iterval betwee testig of 14 moths, which coicided with the child's secod aual screeig visit. DDST Scorig Two differet methods were utilized i the scorig of the DDST. The stadard method (Frakeberg, Dodds, & Fadal, 1973) was used i the descriptive aalysis, comparig outcome o the DDST ad itelligece levels (Tables II ad III). A additioal method of scorig was eeded to facilitate correlatioal aalysis, allowig for the calculatio of age equivalets for each of the four sectors of the DDST. This scorig procedure ivolved idetifyig the highest item passed i a sector (persoalsocial, fie motor-adaptive, laguage, ad gross motor) ad assigig each sector a age-equivalet score by referrig to the horizotal age lie which correspoded to the 50% age rakig of that specific item. These age equivalets (i moths) were averaged across all four sectors to provide a total Dever age equivalet for each subject. These age equivalet data were used i the correlatioal test-retest aalysis.

4 276 Harper ad Wacker Table II. Outcome of Iitial Admiistratio of Dever Developmetal Screeig Test' DDST overall ratig % Abormal 22 2 Questioable 69 7 Normal *l,018 attempted, 55 utestable; = 963. RESULTS/DISCUSSION Of the 1,061 childre who appeared for cliic screeig evaluatios, 1,018 were admiistered the DDST at their iitial cotact. Of these 1,018 childre, 55 were judged utestable due to a variety of factors, such as fatigue, illess, ad ucooperativeess. Utilizig the stadard scorig procedure, the outcome for the remaiig 963 childre revealed that 2% (22) were judged as fallig i the "abormal" category, 7 7o (69) were judged as "questioable," ad 91% (872) received a "ormal" classificatio (Table ID- Although there have bee umerous studies examiig the screeig capability of the DDST (Bares & Stark, 1975; Bryat & Davies, 1974; Bryat, Davies, & Newcombe, 1974; Bryat, Davies, Richards, & Voorhees, 1973; Camp, VaDoorick, Frakeberg, & Lampe, 1977; Frakeberg, Camp, & Va Natta, 1971; Sadier, VaCampe, Rater, Stafford, & Weismar, 1970), o studies have bee doe which report the outcome of the DDST for large samples of preschool childre usig Frakeberg's categories, i.e., abormal, questioable, ad ormal. DDST overall ratig Abormal Questioable Normal Total Table HI. Compariso of DDST Results ad Mea IQ Below IQ S-B (L-M)- ad WPPSI* IQ Staford-Biet (L-M), mea IQ. *Wechsler Preschool Primary Itelligece Scale, mea IQ. 85 ad above IQ Total % 555

5 Efficiecy of the DDST 277 Rather, most authors oly report cross-validated outcome data o smaller preselected samples followig admiistratio of itellectual examiatios. By failig to report the outcomes usig Frakeberg classificatios o large samples, we caot easily evaluate the screeig test's overall efficiecy i estimatig (a) how well a give test reflects base rates of metal retardatio i populatios of childre, ad (b) the subsequet practical utility of a particular screeig test. The latter issue relates to the efficiecy of admiisterig a large umber of screeig tests to a populatio without cosiderig atecedet probabilities or base rates of metal retardatio. These prevalece rates of metal retardatio have a direct affect o the psychometric efficiecy of a screeig test (Nuget, 1976). Put more directly i terms of the preset outcome data, is it efficiet (i ratio of costs to beefits) to admiister 963 evaluatios to locate 22 idividuals, ad does the abormal category outcome of 2% (22/963) compare favorably with kow prevalece rates for metal retardatio? I order to partially aswer these issues, iformatio o itellectual ability is eeded for childre who completed the DDST. A large sample (555) of childre was available (Table III) from the 963 who had completed the DDST. All childre ( = 555) who had bee admiistered both the DDST ad complete itellectual evaluatios (SB-LM or WPPSI) were selected for further aalysis. Admiistratio of itellectual evaluatios was based upo predetermied age ad sex ratios oly. Of the 555 childre screeed, 13 (2%) were classified as abormal. Of these, three were later foud to have itelligece quotiets of 69 or below, seve had itelligece quotiets betwee 70 ad 84, ad three were i the rage of 85 ad above (Table III)- Table IV presets the outcomes of screeig o the DDST with childre who scored 69 ad below ad 85 ad above o itellectual evaluatio ad disregards the questioable category. O the basis of this categorizatio, 36% of the true abormal (IQ< 69) were called abormal (accordig to the DDST), 63% of the true abormal were called ormal (false egative rate), approximately 1% of the true ormal were called abormal (false positive rate), ad 99% of the true ormal were called ormal. Table IV. Metal Retardatio ad DDST Outcome Itelligece quotiet s 69 i 85 Abormal 3 3 Normal 5 437

6 278 Harper ad Wacker O the basis of this presetatio of the data with this sample of disadvataged rural preschool childre, it appears that the major problem for the Dever Developmetal Screeig Test relates to its false egative rate (63%). That is, childre who are later idetified as havig itellectual delays have a higher probability of beig misclassified as ormal ad ot beig referred for subsequet evaluatio or assistace. This would suggest a higher percetage of uderreferrals tha overreferrals whe usig the DDST as a screeig tool with similar populatios of preschool childre. It is clear that whe a 1Q of 69 or below is used as a criterio for idetificatio, the majority of screeig or developmetal report istrumets fare reasoably well. However, as the criterio moves to more subtle learig problems or borderlie itellectual levels, misclassificatios icrease. Further ispectio of the data (Table III) reveals the DDST misclassified approximately 64% of the sample i the rage. O the basis of these data, the DDST is less accurate i idetifyig those preschool-age childre who may have less severe learig ad developmetal delays upo eterig school. However, Camp et al. (1977) emphasize that those subjects receivig a questioable ratig have a higher probability of developig later learig problems ad should i may istaces be referred for further psychological assessmet. Extedig the suggestio of Camp et al. (1977) to cosider referral of questioable ratigs for further psychological assessmet adds aother perspective i viewig the curret data, sice i cliical practice it is likely that both abormal ad questioable ratigs should be referred. We could also logically assume that idividuals with measured ability 1 stadard deviatio below the mea (IQ 84) would be more likely to ecouter learig difficulties. Table V presets the data reflectig a IQ of 84 as a cutoff ad combiig the abormal ad questioable outcomes of the DDST. This presetatio of the data idicates that 39% (37/96) of the true abormal ad questioable were appropriately classified, while 61% (59/%) of the true abormal ad questioable were called ormal (false egative rate); 95% of the true ormal childre were called ormal ad 5% of the true ormal were called abormal (false positive rate). As oted previously, the Table V. Itelligece Level ad DDST Outcome Itelligece quotiet 84 a 85 Abormal ad questioable Normal

7 Efficiecy of the DDST 279 false egative rate is higher tha the false positive rate with this sample, ad suggests that a higher rate of uderreferrals for childre who have ability quotiets 84 ad below is a commo evet. Test-retest reliability data were also computed for 106 childre (Table VI) who were admiistered the DDST o two occasios durig the logitudial screeig project with a average of 14 moths betwee admiistratios. Variatios i the subject frequecy were due to missig data. Mea age at first ad secod admiistratios, respectively, were 43 moths ad 57 moths. Metal age for the group was computed from the Staford-Biet which was admiistered cocurretly with the secod admiistratio of the DDST. The mea IQ of the group ( = 106) was with a stadard deviatio of Correlatios betwee each subtest of the DDST, e.g., PS1/PS2, FM1/FM2, etc., which were admiistered o the average of 14 moths apart, raged from.49 to.56 ad were sigificat at the.002 level. The correlatio betwee the averaged DDST totals (total Dl/total D2) was.70 ad also sigificat at the.002 level. Itercorrelatios of the DDST subtests with the DDST total were higher for cocurret admiistratios, as was to be expected. Itercorrelatios amog each subtest of the DDST for the iitial (youger age, F= 53 moths) admiistratio raged from.48 to.72 (p =.002) ad for the secod admiistratio (older age, x = 57 moths) admiistratio raged from.41 to.70 (p =.002), revealig similar levels of itratest agreemet. Also, the correlatios of each subtest of the DDST ad the total Dever rakig were geerally the same for the iitial (youger) ad secod (older) test admiistratios. Age differeces did ot appear to ifluece these itercorrelatios. Total DDST rakigs for the iitial admiistratio ad the secod admiistratio correlated at similar levels with metal age,.72 ad.74, respectively. Fially, idividual subtest correlatios for the DDST for the iitial ad secod admiistratio were at similar levels ad raged from.50 to.60. I geeral the correlatioal data reflect a adequate level of test-retest reliability for early screeig istrumets with the preschool-age group. Miimal differeces were apparet i the levels of itercorrelatios betwee the age groups examied. Several questios were raised at the outset of this study i relatio to the efficiecy ad practical utility of the DDST with a rural disadvataged group of preschool childre. How does the screeig efficiecy of the DDST correspod to kow or estimated base rates for idetifyig metal retardatio i this populatio? Furthermore, does the use of the DDST eable better idetificatio tha just usig base rates ad iterviews aloe? Before addressig these questios, two issues should be emphasized. Cocurret itellectual measures are oly oe criterio for evaluatig the

8 Table VI. Itercorrelatios Betwee First ad Secod Admiistratio of the DDST ad Metal Age (MA) From thesb-lm' PSI FM1 LAI GM1 TOTD1 MA PS2 FM2 LA2 GM2 TOTD2 PSI FM LAI (102) GM (101) "All correlatios were sigificat at the.002 level. TOTD MA PS FM LA2 GM2 TOTD

9 Efficiecy of the DDST M1 validity of a screeig procedure. Logitudial educatioal ad behavioral performace are also required. Secod, it is very difficult to predict low base rate behaviors with ay brief screeig procedure, i.e., i geeral it is always difficult to predict ucommo evets. Empirical estimates of the prevalece of metal retardatio (base rate) suggest a overall figure at approximately 3.27% for IQs i the 0 to 70 rage (Digma & Tarja, 1960). We might aticipate a slightly higher rate of retardatio give the SES of this group, but this sample does have a relatively high materal educatioal level whe cotrasted to the geeral populatio. Therefore 3.27% was cosidered to be a reasoable estimate. Iterestigly, the percetage of metal retardatio (69 ad below) was approximately 2% based o both stadard itellectual assessmet (11/555) ad DDST classificatio of abormal (22/963). I oe sese this represets a cosesus betwee the DDST ad stadard itellectual measures correspodig to the aticipated prevalece rate of metal retardatio i the geeral populatio. The differece of approximately 1% as estimated by the base rate may reflect the fact that the more severely retarded childre were idetified prior to the study ad were therefore abset from this screeig populatio. Nuget (1976) presets a probability statemet to reflect a aalysis of base rates where the iverse probability that a child who has bee classified as abormal o the DDST will have a IQ below 69 is as follows: P(IQ < 69/AB o DDST) = Apl Apl + Bp2 where A is the base rate of abormality i the screeig populatio (0.327), p\ is the proportio of true abormal childre correctly classified as abormal (.27), B is the base rate of ormality (.95), ad p2 is the proportio of true ormal childre misclassified as abormal (.02). Utilizig the preset study sample data ( = 555), the probability that a child who had bee classified as abormal o the DDST will later obtai a measured IQ of 69 or below is.31. I other words, oly 3 out of 10 childre desigated as abormal o the DDST would receive a IQ of 69 or below. Returig to our study sample data (Table IV) we fid that 3/8 or 36% of the childre who were true abormal were classified appropriately as abormal by the DDST ad 5/8 (63%) were misclassified as ormal by the DDST. These outcome statistics correspod to the aalyses provided by the probability formula offered by Nuget. O the basis of these data, it appears that the desigatio of abormal o the DDST with this sample is accurate 30% of the time. Furthermore, as

10 282 Harper ad Wacker idicated (Table V), the likelihood of false egatives is also a commo evet (61%), resultig i more uderreferrals tha overreferrals. As a result, the overall efficiecy of the DDST remais rather margial. The relatively high percetage of false egatives i this study as cotrasted to other ivestigatios may be attributed to (a) populatio differeces (mostly white, average materal educatio, low SES), (b) age differeces, or (c) examiers beig too liberal i their scorig, although this seems doubtful give the high reliability betwee the first ad secod admiistratio of the DDST. Evaluatio of the cost-beefit ratio of a screeig procedure relates to some extet to the percetage of both uderreferrals (false egatives) ad overreferrals (false positives). If cost ad availability of additioal psychological evaluatios are limited, a low false positive rate would be preferred. O the other had, if the cocer is with maximizig idetificatio of learig/behavioral problems, a low false egative rate would be preferred. However, if the false positive rate becomes too high, the referrig agecy loses creditability. Compromise o these issues is related to (a) availability of resources for evaluatio, ad (b) the impact of uderidetifyig potetial developmetal delays. The screeig outcome of the DDST with the preset sample is cosistet with a situatio which has limited resources for additioal psychological evaluatio. Modificatio of these outcome rates would require adjustmet of cutoff scores depedig upo what is the more desirable outcome. A remaiig questio is whether the DDST is ay more efficiet tha a iterview aloe. This issue was ot formally evaluated i the preset study ad does merit further ivestigatio. Previous studies which have addressed the relatioship betwee paretal estimates of ability/developmetal status ad formal measures have revealed moderate agreemet (Ewart & Gree, 1975) whe behaviorally refereced measures are used but more limited agreemet whe developmetal levels are beig reported retrospectively (Ferald & Lewis, 1976). Whether the DDST is ay more valid tha a iterview aloe awaits ivestigatio. The data preseted suggest that the efficiecy of the DDST is less robust i idetifyig developmetal delays i a rural disadvated preschool group as cotrasted to the authors' (Frakeberg et al., 1973) iitial expectatios. Similar cautios regardig the test's applicability have bee raised by others (Sadier et al., 1970). It is oted that the major problem i screeig childre with these age ad demographic characteristics relates to the test's false egative rate. Childre who are later idetified as havig itellectual delays have a higher probability of beig misclassified as ormal ad ot beig referred for subsequet evaluatio or assistace. Give these data, a higher percetage of uderreferrals is a likely outcome. Correlatioal aalysis reveals adequate test-retest agreemet. The

11 Efficiecy of the DDST 2S3 magitudes of the correlatios are cosistet with similar early screeig istrumets ad the DDST total rakig appears to correlate at a acceptable level with stadard itellectual measures. Fially, as oted by Nuget (1976), evaluatio of screeig istrumets should icorporate some assessmet of the test's efficiecy i relatio to kow or estimated base rates of the behavior of cocer with the specific populatio uder cosideratio. REFERENCE NOTE 1. Holligshead, A. B. Two-factor idex of social positio. Upublished mauscript, (Available from 1965 Yale Statio, New Have, Coecticut.) REFERENCES Bares, K., & Stark, A. The Dever Developmetal Screeig Test: A ormative study. America Joural of Public Health, 1975, 65, Bryat, G., & Davies, K. The effect of sex, social class ad parity o achievemet of Dever Developmetal Screeig Test items i the first year of life. Developmetal Medicie ad Child Neurology, 1974, 16, Bryat, G., Davies, K., & Newcombe, R. The Dever Developmetal Screeig Test: Achievemet of test items i the first year of life by Dever ad Cardiff ifats. Developmetal Medicie ad Child Neurology, 1974,16, Bryat, G., Davies, K., Richards, F. M., & Voorhees, S. A prelimiary study of the use of the Dever Developmetal Screeig Test i a health departmet. Developmetal Medicie ad Child Neurology, 1973, 15, Camp, B., VaDoorick, W., Frakeberg, W., & Lampe, i. Preschool developmetal testig i predictio of school problems. Cliical Pediatrics, 1977,16, Digma, H. F., & Tarja, G. Metal retardatio ad the ormal distributio curve. America Joural of Metal Deficiecy, 1960, 64, Ewart, J. C, & Gree, M. W. Coditios associated with the mother's estimate of the ability of her retarded child. America Joural of Metal Deficiecy, 1957, 62, Ferald, C. D., & Lewis, L. Compariso of Gesell behavior profiles ad mothers'judgmets. Joural of Pediatric Psychology, 1976, /, 46. Frakeberg, W., Camp, B., & Va Natta, P. Validity of the Dever Developmetal Screeig Test. Child Developmet, 1971, 42, Frakebeig, W., Dodds, J., & Fadal, A. Dever Developmetal Screeig Test maual workbook: For ursig ad paramedical persoel. Dever: Uiversity of Colorado Medical Ceter, Nuget, J. A commet o the efficiecy of the revised Dever Developmetal Screeig Test. America Joural ofmetal Deficiecy, 1976,50, Sadier, L., VaCampe, J., Rater, G., Stafford, K., & Weismar, R. Resposes of urba preschool childre to a developmetal screeig test. Joural of Pediatrics, 1970, 77, Thompso, R. J. Cosequeces of usig the 1972 Staford-Biet Itelligece Scale orms. Psychology i the Schools, 1977, 14,