Comparison of the ASQ and PEDS in Screening for Developmental Delay in Primary Care Settings

Transcription

1 Comparison of the ASQ and PEDS in Screening for Developmental Delay in Primary Care Settings Marjolaine M. Limbos, PhD, R. Psych, Department of Psychology, British Columbia Children s Hospital, Vancouver, British Columbia & David P. Joyce, MD, CCFP, Department of Family Practice, University of British Columbia, Vancouver, British Columbia. Running Head/Short Title: Comparison of ASQ and PEDS Key Words: Developmental delay, screening, primary care, well-child visit Financial disclosure: None Conflicts of interest: None Address correspondence to: Dr. Marjolaine M. Limbos, PhD, R. Psych 4428 West 2 nd Ave. Vancouver, BC V6R 1K5 Phone: (604) Fax: (604) info@limboschildpsychology.com

2 ABSTRACT 2 Objectives The purpose of this study was to investigate the sensitivity and specificity of two brief, parentcompleted developmental screening measures, the Ages and Stages Questionnaire (ASQ) and the Parents Evaluation of Developmental Status (PEDS) in children presenting to their primary care provider. Method A sample of 334 children aged months was recruited. Parents completed the ASQ and the PEDS in their home. The presence of 1 predictive concerns or abnormal domains was considered a positive screen. All children underwent evaluation with the criterion measures, administered by a psychologist: the Bayley Scales of Infant Development or the Wechsler Preschool and Primary Scale of Intelligence, the Preschool Language Scale and the Vineland II. Results The mean age of children was 32.4 months. Developmental delay was identified in 34 children (10%). The PEDS had moderate sensitivity (74%) and reasonable specificity (64%). In comparison, the ASQ had significantly higher sensitivity (82%) and specificity (78%). Using 2 predictive concerns on the PEDS or 2 abnormal domains on the ASQ significantly improved specificity of both tests (89% and 94%, respectively) but resulted in very low sensitivity (41% and 47%, respectively). Conclusions

3 3 Comparison of ASQ and PEDS These findings support the guidelines of the American Academy of Pediatrics, demonstrating that both the PEDS and ASQ have reasonable test characteristics for developmental screening in primary care settings. Although the ASQ appears to have higher sensitivity and specificity, the choice of which measure to use should be determined by the practice setting, population served and preference of the physician.

4 INTRODUCTION 4 Developmental delays are common affecting up to 15% of children. 1,2 Recent evidence demonstrating the long term benefits of early intervention has sparked renewed focus on the importance of developmental screening in primary care settings, particularly for preschool children This is reflected in recent guidelines by the American Academy of Pediatrics (AAP) and other authorities who recommend that physicians institute a program of regular developmental screening for young children using standardized screening tools Despite these recommendations, there remains a significant disparity between guidelines and actual practice. Indeed, surveys have demonstrated that a minority of physicians perform routine screening using standardized tools and many children with developmental problems are not identified until they reach school age While there are many possible contributing factors, the lack of screening by physicians may be in part due to inadequate time or remuneration, conflicting reports on the accuracy of available screening tools, as well as the paucity of research that has been conducted in primary care settings. 18, The Parents Evaluation of Developmental Status (PEDS) and Age and Stages Questionnaire (ASQ) are parent-completed developmental screening tools that are increasingly being recommended for use in clinical practice. 11,18, 21, Their popularity is in large part due to several favorable qualities that make them suitable for incorporation into busy practices including: administration by parents, thus requiring little physician time; ease of administration and interpretation; and low cost. 27,28 While there is growing research demonstrating the accuracy of both tests in research settings, generalization of the findings to primary care settings is limited by several factors including: (1) a scarcity of studies in primary care settings; (2) conflicting results for those studies that have used primary care samples; 21,23 (3) failure to administer an adequate criterion measure or failure to administer a criterion measure to

5 all children; 21,23 (4) small sample sizes; and (5) failure to compare various screening tools on the same 5 sample. 34,35 The current study set out to compare the sensitivity and specificity of the ASQ and the PEDS in preschool children presenting to their primary care provider for routine care. The goal was to examine the performance of these tools in the setting for which they are being recommended for use, namely primary care, using a large unselected sample, and administering both screening tools and a clinically relevant criterion measure to all children.

6 6 PATIENTS AND METHODS Study Procedures The current research is part of a larger study examining the accuracy of a number of developmental screening measures, risk factors for developmental delay and physician factors associated with screening in primary care. A convenience sample of children aged months who presented to their primary care provider for routine care were considered eligible for participation. Children were recruited using several methods: (1) direct recruitment by a research assistant from waiting rooms of participating clinics; (2) direct recruitment by receptionists who were trained and provided with a written script for recruitment of children at the time that they checked in for a routine visit; (3) self-referral of parents through posters announcing the study in waiting rooms. Parents agreeing to participate were given an appointment to meet in their homes or at the primary care clinic of one of the investigators (DPJ). Recruitment began in December of 2007 and was completed in September of On the assessment day, a registered child psychologist administered the criterion measures to the child, while parents completed the ASQ and the PEDS in a separate room. Parents and the psychologist were blinded to the results of each others testing. The instructions manuals for the ASQ and PEDS were followed for administration and scoring. 29, 34 Following the psychological assessment, parents were given time with the child and, if necessary, were provided with items (e.g., toys, scissors) required to complete items on the ASQ that they were unsure of and required direct observation of the child. Order of administration of the screening tools was alternated in the research packages at the onset of the study. The PEDS and ASQ were scored by a family physician (DPJ) and the psychologist independently scored all of the criterion measures. Each was blinded to the results of the other s

7 scoring. A summary of the results of the testing was mailed to the child s primary care provider with 7 instructions for follow up. Information on medical and developmental history, ethnic and socioeconomic status was obtained by review of the child s medical record. Parents also completed a non-standardized demographic questionnaire which included their self-reported ethnic, language and socioeconomic background, and a summary of their child s medical, developmental and family history. Participants Children and Parents. A total of 462 children aged months were approached to participate in the study. Children were excluded if there was a prior history of: (1) developmental delay or disability; (2) participation in an intervention program for developmental delay; (3) psychiatric disorder; or (4) birth < 36 weeks gestation. The screening tools and demographic questionnaire were completed mostly by the child s biological mother. While some parents spoke French in the home, all parents reported being fluent in written and spoken English. The study was conducted in communities in northern Ontario, Canada approximately 4 hours drive from the city of Toronto. Participants were said to come from and urban community if they resided in the city of Sudbury (population: 157,857) or North Bay (population: 503,966), Ontario. Those who resided in smaller towns within a 1.5 hour radius of these cities were classified as being rural. Primary Care Providers. Children were recruited from the offices of 80 community-based primary care providers in Ontario, Canada. Seventy participants were family physicians, 7 were nurse practitioners and 3 were pediatricians. Of these primary care providers, 22 were actively involved in a larger study on developmental screening, while the remaining providers simply allowed recruitment of children from their offices.

8 8 Measures Developmental Screening Measures Ages and Stages Questionnaire (ASQ). 30, 33 The ASQ is a brief (15 minute) measure, in which parents rate their child s current skills and development by answering 30 questions covering 5 domains of development including: communication, gross and fine motor, problem-solving, and adaptive skills. A pass/fail score is assigned for each area of development. The form closest to the child s chronological age was used. The presence of any failed domain of the ASQ was considered a positive screen. The ASQ has been validated in large standardization samples of children from diverse ethnic and socioeconomic backgrounds and a moderate sensitivity (70-90%) and specificity (76%-91%) has been reported. 21, 27, 30, 33 As all parents reported being fluent in English, only the English version of the test was administered. Parents Evaluation of Developmental Status (PEDS). 29, 32 The PEDS is a brief, validated, one page developmental screening tool designed for children aged birth to 8 years of age. The tool elicits parent concerns on 10 items across nine domains of development or behavior and takes 2-5 minutes to complete. Response options include yes, no or a little. The presence of one or more predictive/ significant concern was considered a positive screen for developmental delay. The PEDS has a reported moderate sensitivity (74-79%) and specificity (70-80%) and has been shown to perform well in comparison with other developmental screening tests. 11, 21, 28, 29, 37 As all parents reported being fluent in English, only the English version of the test was administered. Criterion Measures for Identification of Developmental Delay

9 The criterion measures for developmental delay in children consisted of a battery of psychological 9 tests of adaptive, cognitive, developmental and language functioning. Although no battery of tests can prove unequivocally that a child has a developmental delay, current practice involves the use of standardized instruments by a registered psychologist. The battery of measures used in the current study is used widely in clinical practice and in prior research on screening for developmental delay and are considered to have excellent reliability and validity. 38, 39 With the exception of the Bayley Scales of Infact Development, criterion measures were administered in English or French based on the language the child spoke most often at home. Bayley Scales of Infant Development Third Edition (BSID-III). 40 The BSID-III is a measure of development for children aged 1 to 42 months. There are 5 subscales: Cognitive, Language, Motor, Social-Emotional, and Adaptive Behavior. To avoid overlap with other tests (see below), language and adaptive behavior scales were not administered. The BSID-III has excellent reliability and validity, and is considered by many to be one of the best measures of infant development It will serve as the primary measure of developmental, cognitive, and motor functioning, in children 12 to 30 months of age as well as for those children over 30 months of age who do not reach basal levels on the Wechsler Preschool and Primary Scales of Intelligence. Wechsler Preschool and Primary Scale of Intelligence Third Edition (WPPSI-III). 42 The WPPSI-III is one of the most widely used tests of cognitive functioning for children above 2 years, 6 months of age. It is composed of 15 tests that are divided into four core and one supplemental test. Additional core subtests are administered to children above age 4. The WPPSI-III provides composite scores of specific domains of intellectual functioning: Verbal Intelligence Quotient (IQ) and Performance

10 10 Comparison of ASQ and PEDS IQ, as well as a Full Scale IQ which represents a child s overall intellectual ability. It has several decades of research to support the clinical utility of the scale, and its correlation with a large number of other scales of intelligence and development supports its validity Vineland Adaptive Behavior Scale-2 nd Edition (VABS-II). 46 The VABS-II is a 297-item, 30 minute, caregiver interview that assesses the adaptive functioning of children between the ages of birth and 19 years. Adaptive behavior is measured in four domains: Communication, Daily Living Skills, Socialization and Motor Skills. An overall Adaptive Behavior Composite is calculated. The VABS-II was standardized using 3,000 individuals, aged newborn to 18 years, has demonstrated reliability and validity and has been used repeatedly to validate developmental screening measures Preschool Language Scale 4 th Edition (PLS-IV). 48, 49 The PLS-IV is a diagnostic instrument for receptive and expressive language disorders in children aged 2 weeks to 6 years of age. It consists of eight receptive/expressive language tasks for each 6 month interval from 2 weeks to 4 years, 11 months and eight tasks for each 12 month interval for ages 5 and 6 years. Administration takes approximately minutes. The scale has been standardized and validated, has demonstrated internal consistency and inter-rater reliability and is used extensively in clinical and research settings to examine language development in preschool children. 48, 49 Definition of Developmental Delay For the purpose of this study, a child was classified as having a developmental delay if he/she scored below the 10 th percentile on any of the criterion measures (i.e., Cognitive, Motor, or Social-Emotional Scale Composite Scores of the BSID-III; Verbal IQ, Performance IQ or Full Scale IQ scores on the WPPSI-III; or

11 Auditory Comprehension, Expressive Communication or Total Language on the PLS-IV) and had a 11 concurrent score below the 10 th percentile on the Adaptive Behavior Composite score of the VABS-II. This criterion is consistent with standards used in research and clinical practice for the identification of developmental and intellectual disability. 38, 39, 44, 47, Data Analysis Differences in characteristics between groups were examined using t tests for continuous variables and χ 2 tests for categorical variables. A series of 2X2 contingency tables were constructed using the number who screened positive or negative on the ASQ or PEDS in the rows and the presence of developmental delay (based on the criterion measures) in the columns. From these tables, sensitivity and specificity were calculated, expressed as percentages with 95% confidence intervals. Values below 70% were classified as low, 70-85% as moderate and >85% as high. 53 Secondary calculations of sensitivity and specificity using the presence of 2 or more predictive concerns on the PEDS or 2 or more abnormal domains on the ASQ as the cut off for a positive screening were also performed. To compare sensitivity and specificity of various screening methods we followed the procedures outlined by Hawass. 54 Matched sample tables were made comparing the PEDS and ASQ results for children with and without developmental delay. McNemar s χ 2 test with Yates correction was then calculated for each matched sample. 54 Cohen s κ was also used to determine level of agreement between screening tests. The DAG_Stat spreadsheet was used for statistical analyses. 55 Results were considered significant at p <0.05. A sample size calculation was performed using a variation of the Log Odds Ratio test. The sample size required to demonstrate a difference of 10% in sensitivity or specificity was 210 participants.

12 12

13 RESULTS 13 A total of 334 children aged months (mean 32.4 months) participated in the study. This represented 72% of children who were initially approached to participate. Reasons for exclusion included: (1) parents not interested in the study (n=58); (2) inability to attend assessment (n=32); (3) outside of age range for inclusion (n =24); or (4) prior diagnosis of developmental delay (n =14). Complete data sets were available for 331 children for the PEDS and 334 children for the ASQ. Table 1 summarizes the remainder of the demographic variables. Children identified with developmental delay were significantly more like to be male gender and come from lower income families. Children with Developmental Delay Thirty-four children (10%) were identified as having a developmental disability using the criterion measure. The primary cause of delay was deemed to be cognitive in 19 children, speech language in 12, and motor 3. More than one developmental delay was present in several children. Of those with cognitive delay, 1 had concurrent speech language delay, and 3 had concurrent motor and speech language delays. One child with motor delay had concurrent speech language delay Agreement between ASQ and PEDS To understand the test characteristics of the PEDS and ASQ, the level of agreement in classifying children with and without developmental delays was examined. Two by two matched sample tables for the ASQ and PEDS are shown in Table 2 for children with and without developmentally delay. As can be seen, overall there was agreement in classification of 206 children (63%). More specifically, there was agreement in classification of 25 (74%) of children with developmental delay (κ=0.38, fair agreement) and 181 (62%) of those without developmental delay (κ=0.02, slight agreement).

14 14 Sensitivity and Specificity of the ASQ and PEDS Table 3 summarizes the results on the accuracy of the ASQ and PEDS. The ASQ had moderate sensitivity (82.3%) and specificity (77%) in screening for developmental delay. The PEDS had moderate sensitivity (74%) but low specificity (64%). There was a significant difference in both the sensitivity and specificity of the ASQ and PEDS. Significant differences were found in the sensitivity and specificity of the following test pairs: PEDS ( 1 Predictive Concern) and ASQ ( 1 Failed Domain); PEDS ( 1 Predictive Concern) and PEDS ( 2 Predictive Concerns); ASQ ( 1 Failed Domain) and ASQ ( 2 Failed Domains). Sensitivity and Specificity Using 2 or More Abnormal Domains or Predictive Concerns Because some clinicians do not refer children for assessment unless 2 or more abnormal domains on the ASQ or 2 or more predictive concerns on the PEDS exist, the test characteristics of the PEDS and ASQ using these alternate cut off points was examined. 20 As can be seen in Table 3, use of these cut off points significantly improved specificity, but sensitivity fell to below acceptable levels for both tests. There was a significant difference between the sensitivity and specificity using the 2 compared to the 1 abnormal cut-off for each test. However, there was no significant difference between the sensitivity or the specificity of the ASQ compared to the PEDS using this cut off.

15 DISCUSSION 15 This study adds to the growing literature supporting the use of the ASQ and PEDS for developmental screening in primary care. While past research has demonstrated the ease of use, acceptability, and validity of these brief screening measures individually, the current study directly compares the sensitivity and specificity of the ASQ and PEDS in primary care settings The main results of this study, that both the ASQ and PEDS have moderate sensitivity (82% and 74%, respectively) accompanied by reasonable specificity (78% and 64%, respectively) for screening for developmental delays, are consistent with findings of past research on conducted on population or standardization samples. 11, 27, 29, 30, 332 The findings strongly support the use of either test for systematic developmental screening, in keeping with the AAP s policy on early detection of children. 11 Our results suggest that the ASQ may have superior sensitivity and specificity to the PEDS. Indeed the higher sensitivity and specificity of the ASQ was not only statistically significant, but also appears to be clinically significant. Of particular concern was the finding that the specificity of the PEDS fell below 70%, the usual cut-off for an acceptable screening measure. 9, 35, 47, 53 Past research on the accuracy of these measures is variable but consistent with our findings, showing a trend toward higher sensitivity and specificity of the ASQ as compared to the PEDS. 11, 21, Despite this, it is important to consider other test factors that will affect actual use by physicians in clinical practice. Indeed, some studies have suggested that the PEDS may have some advantages over the ASQ, primarily related to shorter administration time and slightly lower cost. 28 Thus, we conclude that while the ASQ appears to have the optimal combination of sensitivity and specificity, both the ASQ and PEDS are acceptable for developmental screening.

16 To date there has been little research directly comparing the accuracies of developmental 16 screening tools in primary care settings. Rydz and colleagues studied a sample of 248 children presenting to their pediatrician for an 18 month well child visit, and assigned children to be screened with either the ASQ or the Child Development Inventory (CDI). 23 A single developmental inventory (Batelle Developmental Inventory) was used as the criterion measure and was only administered to those who tested positive on the screening test. Their findings suggested that both measures had low sensitivity (67% and 50%, respectively) and only the CDI had acceptable specificity (86% versus 39% for ASQ). The interpretation of these findings is significantly limited by the failure to administer the criterion measure to all children (i.e., making determination sensitivity and specificity problematic), use of a single developmental inventory as the criterion measure, and having a significant lag time between the administration of the screening test and criterion measure. Sices and colleagues recently compared the level of agreement between the PEDS and the ASQ in a sample of 60 preschool children. 21 Their findings suggested that the two tests had only fair overall agreement, with discordant classification of children 1/3 of the time. Consistent with their findings, we found disagreements in classification of just over one third of children, and only slight to fair agreement between the PEDS and ASQ. Disagreement in classification was higher among those without a developmental delay (39%) than for those with a developmental delay (27%). Although informative in describing differences between the two tests, the study by Sices was limited by the absence of a criterion measure making it impossible to compare the ability of the tests to correctly classify children as developmentally delayed or not. Thus, the current study addresses many of the limitations of prior comparative studies of developmental screening tests by using a larger sample size, administering both screening tests and criterion measures to all children, using a comprehensive psychological evaluation as the criterion measure, and having no time lag between administration of criterion and screening

17 measures. 35 As such, this study provides clinicians with valuable information about the ability of the 17 tests to correctly classify children with developmentally delay in primary care settings. While the manuals for the ASQ and PEDS both recommend use of a single abnormal domain or predictive concern as the cut off for referral, many pediatricians in practice choose not to initiate further assessment unless a child has 2 or more abnormals. 21, 29, 32, 33 The findings of the current strongly support the use of a single domain/predictive concern cut off point. Although use of a more stringent cutoff improved the specificity of the tests dramatically (89% for PEDS and 94% for ASQ), sensitivity fell to below 50% for both tests and is clearly inadequate for a screening test. Pediatricians in practice should be aware that selecting only children who score abnormal in 2 or more areas will result in significant under referral of children with developmental delay. There are several limitations to the current study. Although an attempt was made to replicate the real world clinical application of the PEDS and ASQ by recruitment of children from primary care offices, administration of screening tests by parents at home, and interpretation of results by a family physician, all aspects were overseen and carried out by members of the research team. Prior studies have already, however, confirmed the potential for implementing similar screening strategies in real world settings. 24, 25 Secondly, while we used a valid and reliable criterion measure, assessment took place at one point in time and ideally longitudinal assessments would be conducted to confirm that the developmental delay is stable. 35 Finally, while the sample size was large enough to demonstrate differences between the screening tests, the absolute numbers of children with developmentally delay was relatively small and a larger sample size would be ideal to give support to the conclusions.

18 CONCLUSIONS 18 The findings add support the guidelines of the AAP, demonstrating that both the PEDS and ASQ have reasonable test characteristics for developmental screening in primary care settings. Although the ASQ appears to have higher sensitivity and specificity, the choice of which measure to use should be determined by the practice setting, population served and preference of the physician. Guided by the favorable results of the current study, future research should replicate this study engaging actual primary care physicians in the administration, scoring, interpretation, and follow up to better understand the performance of these tests in real world clinical practice.

19 ACKNOWLEDGMENTS 19 We thank the parents and primary care providers who participated in the study and generously contributed their time. Special thanks to the staff and nurses at Shkagamik-Kwe Health Centre for their assistance and support of this project, and to Lise Levac for coordinating many aspects of the study.

20 REFERENCES Boyle CA, Decoufle P, Yeargin-Allsopp M. Prevalence and health impact of developmental disabilities in US children. Pediatrics. 1994;93(3): Yeargin-Allsopp M, Boyle C. Overview: the epidemiology of neurodevelopmental disorders. Ment Retard Dev Disabil Res Rev. 2002;8(3): Anderson LM, Shinn C, Fullilove MT, et al. The effectiveness of early childhood development programs: a systematic review. Am J Prev Med. 2003;24(suppl 3):S32 S Berlin LJ, Brooks-Gunn J, McCarton C, McCormick MC. The effectiveness of early intervention: examining risk factors and pathways to enhanced development. Prev Med. 1998;27(2): Hill JL, Brooks-Gunn J, Waldfogel J. Sustained effects of high participation in an early intervention for low-birth-weight premature infants. Dev Psychol. 2003;39(4): Chamberlin RW, Szumonwski EK, Zastowny TR. An evaluation of efforts to educate mothers about child development in pediatric office practices. Am J Public Health. 1979; 69: Casey PH, Whitt JK. Effect of the pediatrician on the mother-infant relationship. Pediatrics. 1980;65: McCain M, Mustard F. Early Years Study. Toronto, ON: Government of Ontario; 1999.

21 21 9. Schweinhart LJ, Montie J, Xizng Z, et al. Lifetime Effects: The High/Scope Perry Preschool Study Through Age 40. Ypsilanti, MI: HighScope Press; Drillien CM, Pickering RM, Drummond MB. Predictive value of screening for difficult areas of development. Dev Med Child Neurol. 1988;30: American Academy of Pediatrics, Council on Children With Disabilities, Section on Developmental Behavioral Pediatrics, Bright Futures Steering Committee, Medical Home Initiatives for Children With Special Needs Project Advisory Committee. Identifying infants and young children with developmental disorders in the medical home: an algorithm for developmental surveillance and screening [published correction appears in Pediatrics. 2006;118(4): ]. Pediatrics. 2006;118(1): Ontario Children s Health Network/Ontario College of Family Physicians. Getting it Right at 18 Months Making it Right for a Lifetime: Report of the Expert Panel on the 18 Month Well Baby Visit. Toronto, ON; September, Best Start Expert Panel on Early Learning. Early Learning for Every Child Today: A framework for Ontario early childhood settings. Toronto, ON; December, Wiggins LD, Baio J, Rice C. Examination of the time between first evaluation and first autism spectrum diagnosis in a population-based sample. J Dev Behav Pediatr. 2006;27(suppl 2):S79 S87.

22 15. Tomblin JB, Records NL, Buckwalter P, Zhang X, Smith E, O'Brien M. Prevalence of specific 22 language impairment in kindergarten children. J Speech Lang Hear Res.1997;40(6): Glascoe, FP, Shapiro, HL. Introduction to developmental and behavioral screening. (Available at: accessed September 2010). 17. Janus, M. Measuring community early child development. The CAP Journal. 2006;14(3): Sices L, Feudtner C, McLaughlin J, Drotar D, Williams M. How do primary care physicians identify young children with developmental delays? A national survey. J Dev Behav Pediatr. 2003;24 (6): Sand N, Silverstein M, Glascoe FP, Gupta VB, Tonniges TP, O'Connor KG. Pediatricians reported practices regarding developmental screening: do guidelines work? Do they help? Pediatrics. 2005;116 (1): Limbos MM, Joyce DP, Roberts GJ. Nipissing District Developmental Screen. Patterns of use by physicians in Ontario. Can Fam Physician. 2009;56;e66-e Sices, L, Stancin, T, Kirchner, L, Bauchner, H. PEDS and ASQ developmental screening tests may not identify the same children. Pediatrics. 2009;124;e640-e647.

23 22. Feightner JW, for the Canadian Task Force on the Periodic Health Examination. Preschool 23 screening for developmental problems. In: Ministry of Supply and Services, eds. The Canadian Guide to Clinical Preventive Health Care. Ottawa, Canada: Ministry of Supply and Services; Rydz D, Srour M, Oskoui M, et al. Screening for developmental delay in the setting of a community pediatric clinic: a prospective assessment of parent-report questionnaires. Pediatrics. 2006;118 (4):e1178-e Hix-Small H, Marks K, Squires J, Nickel R. Impact of implementing developmental screening at 12 and 24 months in a pediatric practice. Pediatrics. 2007;120(2): Earls MF, Hay SS. Setting the stage for success: implementation of developmental and behavioral screening and surveillance in primary care practice the North Carolina Assuring Better Child Health and Development (ABCD) Project. Pediatrics. 2006;118(1):e183-e Schonwald A, Huntington N, Chan E, Risko W, Bridgemohan C. Routine developmental screening implemented in urban primary care settings: more evidence of feasibility and effectiveness. Pediatrics. 2009;123(2): Squires J, Bricker D, Potter L. Revision of a parent-completed development screening tool: Ages and Stages Questionnaires. J Pediatr Psychol. 1997;22(3):

24 24 Comparison of ASQ and PEDS 28. Brothers KB, Glascoe FP, Robershaw NS. PEDS: Developmental Milestones- An accurate brief tool for surveillance and screening. Clin Pediatr (Phila). 2008;47(3): Glascoe FP. Parents Evaluation of Developmental Status. Nashville, TN: Ellsworth & Vandermeer Press, Ltd; Bricker D, Squires J. Ages & Stages Questionnaires: A Parent-Completed, Child-Monitoring System. 2nd ed. Baltimore, MD: Paul H. Brookes Publishing Co; Glascoe FP. Evidence-based approach to developmental and behavioural surveillance using parents concerns. Child Care Health Dev. 2000;26(2): Glascoe FP. Collaborating With Parents: Using Parents Evaluations of Developmental Status to Detect and Address Developmental and Behavioral Problems. Nashville, TN: Ellsworth & Vandermeer Press LLC; Squires J, Potter L, Bricker D. The ASQ User's Guide for the Ages & Stages Questionnaires: A Parent-Completed, Child-Monitoring System. 2nd ed. Baltimore, MD: Paul H. Brookes Publishing Co; Marks K. Should general pediatricians not select the Ages & Stages Questionnaire in light of the Rydz et al. study? Pediatrics. 2007;120(2):

25 35. Marks K, Glascoe FP, Aylward GP, Shevell MI, Lipkin PH, Squires JK. The thorny nature of 25 predictive validity studies on screening tests for developmental-behavioral problems. Pediatrics. 2008; 122(4); Glascoe FP. Parents concerns about children's development: prescreening technique or screening test? Pediatrics. 1997;99(4): Glascoe FP, MacLean WE, Stone WL. The importance of parents concerns about their child's behavior. Clin Pediatr (Phila). 1991;30(1): Sattler JM. Assessment of Cchildren. Revised and Updated Third Edition. San Diego, CA: Jerome Sattler; Anastasia A. Psychological Testing. New York, NY: Macmillan; Bayley N. Bayley Scales of Infant Developmental. 3rd ed. Toronto, Canada: Psychological Corporation; Dahinten, SV, Ford, L. Validation of the Nipissing District Developmental Screen foruse with Children and Toddlers Working Paper. Vancouver, BC: Human Early Learning Partnership (HELP); 2004.

26 42. Weschler D. Weschler Preschool and Primary Scale of Intelligence. 3rd ed. San Antonio, TX: 26 Harcourt Assessment, Inc.; McFie, J. Assessment of Organic Intellectual Impairment. Oxford, England: Academic Press; Beres KA, Kaufman AA, Perlman MD. Assessment of child intelligence. In: Goldstein G, Hersen M, eds. Handbook of Psychological Assessment. 3rd ed. Kidlington, UK: Elsevier Science Ltd.; 2000: Halperin JM, McKay KE. Psychological testing for child and adolescent psychiatrists: a review of the past 10 years. J Am Acad Child Adolesc Psychiatry. 1998;37: Sparrow SS, Cicchetti DV, Balla DA. Vineland Adaptive Behavior Scales (Second Edition) Survey Inteview Form Manual. Minneapolis, MN: NCS Pearson Inc.; Glascoe FP, Byrne KE, Ashford LG, Johnson KL, Chang B, Strickland B. Accuracy of the Denver II in developmental screening. Pediatrics. 1992;89: Zimmerman IL, Steiner VG, Pond RE. Preschool Language Scale. 4 th ed. San Antonio, TX: Psychological Corporation; Zimmerman IL, Castilleja NF. The role of a language scale for infant and preschool assessment. MRDD Research Review. 2005;11(3):

27 Robins DL, Fein D, Barton M, Green J. The Modified Checklist for Autism in Toddlers: An initial study investigating the early detection of autism and pervasive developmental disorders. J Autism Dev Disord. 2001;31: Robins DL, Fein D, Barton M. Modified Checklist for Autism in Toddlers (MCHAT) Follow up Interview. Robins, Fein, Barton, American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4 th ed. text rev. Washington, DC: Author; American Psychological Association. Standards for Educational and Psychological Tests. Washington, DC: Author; Hawass, NED. Comparing the sensitivities and specificities of two diagnostic procedures performed on the same group of patients. Br J Radiol. 1997;70(832): Mackinnon, A. A spreadsheet for the calculation of comprehensive statistics for the assessment of diagnostic tests and inter-rater agreement. Comput Biol Med. 2000;30(3):

28 28