FINAL PROGRESS REPORT FOR VALIDITY WITHOUT A GOLD STANDARD

Transcription

1 FINAL PROGRESS REPORT FOR VALIDITY WITHOUT A GOLD STANDARD ORIGINAL AIMS 1. Analyze existing data sets to identify promising screens. Using the ROC (Receiver Operating Characteristic) techniques, we propose to use existing data sets to provide unbiased estimates of efficiency for different screens relative to diagnostic criteria. We will find studies that used a structured diagnostic instrument of known reliability along with a screening instrument. The crosscalibration of screens and validators will generate a network of estimates that can be used to establish the best screens in different settings, for different diagnoses and demographic groups. The effects of prevalence, screen cut-point, and cost will be incorporated. 2. Test the screens using three diagnostic interviews as criteria. We propose to begin implementing findings from the first stage. Because the validity of screens must be evaluated separately in each setting in which they are widely used, we propose to start with a primary care pediatric setting for 3 reasons: (1) major public health significance; (2) wide generalizability; and (3) can be studied within the budgetary constraints of this RFA. We will recruit 1200 parent-child pairs (children ages 9-16) from Duke Pediatric Clinics. The sample will be stratified by sex, ethnicity (White, African American), and age (9-12, 13-16). Each subject pair will receive, at random, two (2) state-of-the-art diagnostic interviews, done about one (1) week apart (1200 x 4 = 4800 total interviews), using the sections for depression, anxiety, conduct disorder, oppositional disorder, ADHD, and substance abuse. Three interviews will be used pair-wise (DISC-IV/DAWBA, DAWBA/CAPA, DISC-IV/CAPA). The QCROC (quality ROC) approach will be used to generate estimates of quality- and cost-adjusted predictive power of the screens relative to the three diagnostic interviews, adjusting the variance estimates for intra-subject dependency. 3. Generate a unique data set for comparing 3 approaches to diagnosis: respondent-based (DISC-IV), interviewer-based (CAPA), and best estimate (DAWBA). We will examine the performance of each relative to a nomological net of validators: school performance, service use, impact on family, functional impairment, family psychiatric history, maternal depression, etc. STUDIES AND RESULTS In year 1 of the study, time was spent organizing the existing data sets into large databases suitable for ROC meta-analyses, and developing advanced statistical modeling needed for analyzing these data sets for the identification of candidate screening instruments. In year 2, we began recruiting subjects and conducting interviews using the selected screens from year 1, and DISC, CAPA and DAWBA diagnostic interviews. In years 4 and 5, we continued to recruit subjects, conduct interviews, and analyze data. We have currently reached our targeted number of recruits, as described in the attached inclusion enrollment report.

2 1. Organizing data sets: We collected nine data sets from the first year of the study, which are converted to SAS data sets. We finished the development of database conversion programs for retrieval and processing of our data stream. Subject and data tracking is now accomplished through the use of a custom client/server computer application. The front end is written in Delphi (Borland) and the backend is in Oracle. The program assigns subjects to a cell in the stratification chart and tracks the subject s status. It has a page for creating custom queries and also provides reports of overall study progress. Custom double-entry applications for the screens and the CAPA have been written using a web accessible Flash (Macromedia) user interface. The data storage is set up using a MYSQL system. We also developed an automated system to detect interviews uploaded from interviewers' laptops to the server, write and run a DBMSCopy batch file to convert DISC interviews stored in 48 MS Access tables to SAS datasets, write and run a DBMSCopy batch file to convert DAWBA interviews stored as 49 ASCII files to SAS datasets, and write and run a SAS program to merge separate data tables for both the DISC and DAWBA interviews. The automated system allows the conversion of DISC ASCII files, DAWBA Microsoft Access files, and CAPA MYSQL files into a SAS data base by utilizing the build-in data base conversion programs available in the current version of SAS (version 9.1.3), including the newly implemented MYSQL-to-SAS data conversion engine. During last year and early this year, we modified the tracking system back-end to be compatible with other in-house systems. We also expanded the reporting and querying capabilities of the user interface, and improved the Flash/PHP/MySQL data-entry system. 2. Development of new statistical methods and analyses of preliminary data: We developed novel Bayesian methods for robust ROC estimation that accounts for heterogeneity among sensitivities, specificity s of the diagnostic tests and when there is no gold standard diagnostic test. Specifically, we investigated Berkson and classical measurement error structures to account for imperfect gold standard. Results show that our ROC approach performs quite robustly even if the true diagnosis (e.g., depression) is an imperfect gold standard. We have also conducted extensive ROC analyses using the data processed so far. Dr. Erkanli and Dr. Costello presented a report of early findings to NIMH during a conference call in April This report is based on the current data consisting of 731 child/parent pairs who completed at least one CAPA 1 month or CAPA 3 month (or both) instruments during the first or second interview. It presents preliminary statistical analysis involving SDQ, CBCL (parent versions only) and CAPA 1 month and 3 month diagnoses and scales scores on five major psychopathology diagnostic groups (depression, anxiety, conduct, adhd, and opp). A second set of analysis were performed upon a request from NIMH to further investigate the predictive performance of a sub-screening scale

3 constructed using five items from SDQ questionnaire. Recently, we have also compared DAWBA and CAPA instruments using SDQ and short SDQ subscreening scale for predicting any DSM-IV diagnosis. These findings can be found in the appendix. Currently, we are assessing the predictive ability of short SDQ screening and four additional questions from DAWBA in predicting serious emotional disorders (SED). 3. Publications: We published a paper utilizing Bayesian ROC methodology, titled A Bayesian non-parametric analysis of ROC curves in the November 2006 issue of Statistics in Medicine journal. Drs Erkanli and Costello are currently working on a research paper to present preliminary findings for comparing CBCL and SDQ, and short SDQ screens in predicting any psychiatric diagnosis and SED. 5) Recruitment and Interviewing: We have recruited 1270 subjects from late October 2003 until the present. We have completed 1250 pairs of parent-child interviews (1250 subjects x 4 interviews for a total of 5000 interviews). We have 3 parent/child interviews scheduled for a future date (at the time this report was created). We have a total of 4 subjects who have refused to complete the 2 nd interview. The parents did not have the time to complete the second interview and asked not to be contacted again. We have developed a data entry system, and we are currently entering data. Some preliminary data analysis has been done. 6) We are developing a web based data retrieval program that will allow other investigators to access statistical findings and data bases relevant to their research. SIGNIFICANCE The significance of this study lies in the fact that a battery of cost efficient screening tests brings greater benefits than would further work on expensive diagnostics. This study provides such tests by utilizing information from studies that have different diagnostic styles. FUTURE PLANS The funding for this grant ended at the end of July We are currently processing data collected so far, and conducting relevant ROC analyses. Our consultant Dr. Goodman in England has recently completed scoring of DAWBA sections of the study. Additionally, diagnostic algorithms are being developed to score DISC section of the study. We are working on several papers to communicate the early findings that were requested by NIMH.

4 PAPERS Published Erkanli A, Sung M, Costello EJ, Angold A (2006). A Bayesian non-parametric analysis of ROC curves Statistic in Medicine, 25, Erkanli, D. D. Taylor, D. Dean, F. Eksir, D. Egger, J. Geyer, B. H. Nelson, B. Stone, H. A. Fritsche, and R. B.S. Roden (2006) Application of Bayesian modeling of autologous antibody responses against ovarian tumor-associated antigens to cancer detection. Cancer Research, 66, Phillips, S. D., Erkanli, A., Costello, E. J., & Angold, A. (2007). Differences among children whose mothers have a history of arrest. Women in Corrections, 17(2/3), Ryan A. Brown, Carol M. Worthman, E. Jane Costello, Alaattin Erkanli (2007). The Life Trajectory Interview for Youth (LTI-Y): method development and psychometric properties of an instrument to assess life-course models and development. Int.J. Methods Psyhicatr. Res. 15(4), E. J. Costello, A. Erkanli, and A. Angold (2007). Is there an epidemic of child and adolescent depression? Journal of Child Psychology and Psychiatry. Online Early Issue. Goldston, D., Walsh, A., Arnold, E., Reboussin, B., Daniel, S., Erkanli, A., Nutter, D., Hickman, E., Palmes, G., Snider, E., Wood, F. (2007). Reading problems, psychiatric disorders, and functional impairment from mid- to late adolescence. Journal of the American Academy of Child and Adolescent Psychiatry. In press.

5 APPENDIX 1 A report of preliminary data analysis of Validity grant by Al Erkanli and Jane Costello This report presents preliminary statistical analysis involving SDQ, CBCL (parent versions only) and CAPA 1 month and 3 month diagnoses and scales scores on five major psychopathology diagnostic groups (depression, anxiety, conduct, adhd, and opp). The representative data set used in these analyses consist of 731 child/parent pairs who completed at least one CAPA 1 month or CAPA 3 month (or both) instruments during the first or second interview. By design every child and parent pairs received both CAPA 1 month and 3 month at either of the first or second interview due to randomization. Out of 731 child-parent pairs, 362 completed CAPA 1 month, and 369 completed CAPA 3 month, respectively (see table below where wave=1 refers to first, and wave=2 refers to second interview, respectively). Table1.Distribution of number of CAPA interviews by order of interview (wave) study wave Frequency Percent Row Pct Col Pct 1 2 Total ƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆ CAPA1mo ƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆ CAPA3mo ƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒˆ Total

6 Currently, Validity study has completed 1252 child-parent pairs, and we are interviewing the remaining 13 child-parent pairs to finish the Validity interview process. This report provides three types of analyses: 1) ) Test-retest reliability of CBCL and SDQ screens. 2) Basic statistical (mean, range, etc) and correlation analyses involving CBCL, SDQ screening scores, CBCL_EXT (externalizing), CBCL_INT (internalizing), SDQ_EMO (emotional), and SDQ_BEH (behavioral) sub screening scores, and DSM IV scales scores of depression, conduct, anxiety, adhd, and opp. 3) ROC analyses of CBCL, SDQ screening scores, CBCL_EXT (externalizing), CBCL_INT (internalizing), SDQ_EMO (emotional), and SDQ_BEH (behavioral) sub screening scores, and DSM IV binary diagnoses of depression, conduct, anxiety, adhd, and opp disorders, a well as binary diagnosis of externalizing and internalizing disorders. For each diagnostic outcome and each diagnosis, ROC analyses provide optimal screening cut-off points, and the corresponding estimates of sensitivity, specificity, PVP, and PVN, and kappa statistic, corresponding to the optimal screening cut-offs. As the optimality criterion to select the screen cut-off, we used the maximum Kappa statistics as outlined in Helena Kraemer s book Evaluating Medical Tests, which is the famous Cohen s Kappa statistics. We computed the ROC curves for each screen-diagnosis pair. For the purpose of this report, we will display only a few of the basic ROC curves, but all the ROC curves for each screen-diagnostic pair are available from Dr. Erkanli upon request. We also have the SAS programs to compute the optimal cut-offs and ROC curves and will send them to NIMH if need arises. Cost of CBCL and SDQ As is well known, SDQ (parent version) has about 22 items that CBCL has 125. It takes about 5-7 minutes to complete SDQ versus minutes to complete CBCL (parent versions) during telephone interview. So, SDQ is more time efficient and cheaper to implement. If SDQ provides efficiency measures (sensitivity, specificity, PVP, PVN, etc) similar, but not necessarily identical, to those obtained from CBCL then we can make the argument that SDQ is a cost effective screening instrument in predicting majority of psychiatric disorders in the population. This report will show that SDQ is slightly less efficient than CBCL in a majority of diagnostic areas, but not too far behind and it performs better than CBCL in some other areas.

7 Reliability In Validity study, the parents and children have completed SDQ and CBCL screens in both first and second interviews which provided measures of internal consistency of CBCL and SDQ. Using a random-effects model implemented in SAS PROC MIXED, the reliability of SDQ and CBCL were measured by the Intra Class Correlation Coefficient (ICC) as follows: ICC(CBCL) = 0.86; ICC(SDQ) = These are very close. Summary statistics and correlations Below are summary measures of diagnoses, screens and DSM IV scale scores. Table 2. Prevalences of major diagnosis: CAPA 1month. Any of 5 DSM4 Diagnosis Cumulative Cumulative j4_dx Frequency Percent Frequency Percent ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ major depressive episode/dsm-4 Cumulative Cumulative j4_dep Frequency Percent Frequency Percent ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ Joint DSM-IV CD with duration Cumulative Cumulative j4l_cd Frequency Percent Frequency Percent ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ Any anxiety=1, 0 otherwise Cumulative Cumulative j4l_anx Frequency Percent Frequency Percent ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ ODD with duration and impairment Cumulative Cumulative j4li_op Frequency Percent Frequency Percent ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ

8 ADHD w/ ons+imp+<7yo start Cumulative Cumulative p4lo_hy Frequency Percent Frequency Percent ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ Internalizing (depression+ anxiety) Cumulative Cumulative J4_emo Frequency Percent Frequency Percent ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ Externalizing (cd+adhd+opp) Cumulative Cumulative J4_beh Frequency Percent Frequency Percent ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ Table 3. Prevalences of major diagnosis: CAPA 3 month. Any of 5 DSM4 Diagnosis Cumulative Cumulative j4_dx Frequency Percent Frequency Percent ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ major depressive episode/dsm-4 Cumulative Cumulative j4_dep Frequency Percent Frequency Percent ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ Joint DSM-IV CD with duration Cumulative Cumulative j4l_cd Frequency Percent Frequency Percent ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ

9 Any anxiety=1, 0 otherwise Cumulative Cumulative j4l_anx Frequency Percent Frequency Percent ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ ODD with duration and impairment Cumulative Cumulative j4li_op Frequency Percent Frequency Percent ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ ADHD w/ ons+imp+<7yo start Cumulative Cumulative p4lo_hy Frequency Percent Frequency Percent ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ Internalizing (depression+ anxiety) Cumulative Cumulative J4_emo Frequency Percent Frequency Percent ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ Externalizing (cd+adhd+opp) Cumulative Cumulative J4_beh Frequency Percent Frequency Percent ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ

10 Correlations Correlations are also given in appendices 1 and 2 due to space limitations. The cross correlations between CBCL and SDQ within CAPA 1 month and within CAPA 3 month interviews were respectively 0.82 (p-value<0.0010) and 0.85 (pvalue<0.0001). Overall, both CBCL and SDQ (and sub screening scales for internalizing and externalizing problems) were significantly correlated with total DSM IV scale scores, and number of depressive, anxiety, conduct, odd, and adhd symptoms. Both correlations were also similar in magnitude. For example, the correlation between CBCL and the number of total DSM IV symptoms was 0.62, and the correlation between SDQ and the number total DSM IV symptoms was 0.55, respectively. These bivariate correlations suggest that SDQ was associated with the symptom scales similar to CBCL, if not better. ROC analysis Here we present the ROC analysis for each of the diagnostic-screen pair in CAPA 1 month and CAPA 3 month interviews. Table 4- Table 11 give the optimal cut off points for each screen in predicting the binary DSM IV diagnoses, and the associated efficiency measures. Table 4. Any of 5 DSM4 Diagnosis. SCREENS Optimal Cut off Specificity Sensitivity PVP PVN Kappa 1M #3M 1M 3M 1M 3M 1M 3M 1M 3M 1M 3M CBCL SDQ SDQ_EMO SDQ_BEH CBCL_INT CBCL_EXT Table 5. Major depressive episode/dsm-iv. SCREENS Optimal Cut off Specificity Sensitivity PVP PVN Kappa 1M #3M 1M 3M 1M 3M 1M 3M 1M 3M 1M 3M CBCL SDQ SDQ_EMO CBCL_INT

11 Table 6. Joint DSM-IV CD with duration SCREENS Optimal Cut off Specificity Sensitivity PVP PVN Kappa 1M #3M 1M 3M 1M 3M 1M 3M 1M 3M 1M 3M CBCL SDQ SDQ_BEH CBCL_EXT Table 7. Any anxiety. SCREENS Optimal Cut off Specificity Sensitivity PVP PVN Kappa 1M #3M 1M 3M 1M 3M 1M 3M 1M 3M 1M 3M CBCL SDQ SDQ_EMO CBCL_INT Table 8. ODD with duration and impairment. SCREENS Optimal Cut off Specificity Sensitivity PVP PVN Kappa 1M # 3M 1M 3M 1M 3M 1M 3M 1M 3M 1M 3M CBCL SDQ SDQ_BEH / CBCL_EXT Table 9. ADHD. SCREENS Optimal Cut off Specificity Sensitivity PVP PVN Kappa 1M # 3M 1M 3M 1M 3M 1M 3M 1M 3M 1M 3M CBCL SDQ SDQ_BEH CBCL_EXT

12 Table 10. Internalizing (depression+ anxiety) SCREENS Optimal Cut off Specificity Sensitivity PVP PVN Kappa 1M #3M 1M 3M 1M 3M 1M 3M 1M 3M 1M 3M CBCL SDQ SDQ_EMO CBCL_INT Table 11. Externalizing (cd+adhd+opp) SCREENS Optimal Cut off Specificity Sensitivity PVP PVN Kappa 1M #3M 1M 3M 1M 3M 1M 3M 1M 3M 1M 3M CBCL SDQ SDQ_BEH CBCL_EXT The ROC curves below provide a direct visual comparison of the performance of CBCL and SDQ. For example, in Figure 1, the AUC (area under the curve) for CBCL- any diagnosis (CAPA 1 month) pair was 0.82, compared to AUC of 0.79, in Figure 2, for SDQ-any diagnosis pair (CAPA 1 month). The shapes of these ROC curves are again very similar. Figures 3 and 4 display the ROC curves for CBCL-any diagnosis and SDQ-any diagnosis in CAPA 3 month instrument, respectively. Although not shown here, all the ROC analyses suggested similarities between CBCL and SDQ in terms of the shape of the ROC curves and associated AUCs. Figure 1. ROC curve for CBCL- any Dx: CAPA 1 month.

13 Figure 2. ROC curve for SDQ-any Dx: CAPA 1 month.

14 Figure 3. ROC curve for CBCL- any Dx: CAPA 3 month.

15 Figure 4. ROC curve for SDQ-any Dx: CAPA 3 month.

16 CAPA 1 month summary statistics and correlations. Simple Statistics Variable N Mean Std Dev Sum Minimum Maximum PTotalN pebdtot pemotion pbehav PInternN PExternN j4ndx j4ndep j4ncd j4nanx j4nopp p4nadh Variable Label PTotalN P-CBCL total problems pebdtot SDQ total difficulties scale pemotion SDQ emotional symptoms scale adjusted for missing pbehav SDQ joint total external difficulties score

17 PInternN P-CBCL internalizing PExternN P-CBCL externalizing j4ndx number of 5 DSM4 diagnoses j4ndep number of depressive symptoms j4ncd Number of DSM-IV CD symptoms j4nanx ANX: symptom scale score j4nopp Number of DSM-IIIR ODD symptoms p4nadh # of ADHD symptoms Pearson Correlation Coefficients Prob > r under H0: Rho=0 Number of Observations PTotalN pebdtot pemotion PTotalN P-CBCL total problems <.0001 <.0001 pebdtot SDQ total difficulties scale <.0001 <.0001 pemotion SDQ emotional symptoms scale adjusted for missing <.0001 <.0001 pbehav SDQ joint total external difficulties score <.0001 <.0001 <.0001 PInternN P-CBCL internalizing <.0001 <.0001 <.0001 PExternN P-CBCL externalizing <.0001 <.0001 <.0001 j4ndx number of 5 DSM4 diagnoses <.0001 <.0001 <.0001 j4ndep number of depressive symptoms <.0001 <.0001 < j4ncd Number of DSM-IV CD symptoms <.0001 <.0001 < j4nanx ANX: symptom scale score <.0001 <.0001 < j4nopp Number of DSM-IIIR ODD symptoms <.0001 <.0001 <.0001 p4nadh # of ADHD symptoms <.0001 <.0001 <.0001

18 pbehav PInternN PExternN PTotalN P-CBCL total problems <.0001 <.0001 <.0001 pebdtot SDQ total difficulties scale <.0001 <.0001 <.0001 pemotion SDQ emotional symptoms scale adjusted for missing <.0001 <.0001 <.0001 pbehav SDQ joint total external difficulties score <.0001 <.0001 PInternN P-CBCL internalizing <.0001 <.0001 PExternN P-CBCL externalizing <.0001 <.0001 j4ndx number of 5 DSM4 diagnoses <.0001 <.0001 <.0001 j4ndep number of depressive symptoms <.0001 <.0001 < j4ncd Number of DSM-IV CD symptoms <.0001 <.0001 < j4nanx ANX: symptom scale score <.0001 <.0001 < j4nopp Number of DSM-IIIR ODD symptoms <.0001 <.0001 <.0001 p4nadh # of ADHD symptoms <.0001 <.0001 <.0001 j4ndx j4ndep j4ncd PTotalN P-CBCL total problems <.0001 <.0001 < pebdtot SDQ total difficulties scale <.0001 <.0001 <

19 pemotion SDQ emotional symptoms scale adjusted for missing <.0001 <.0001 < pbehav SDQ joint total external difficulties score <.0001 <.0001 < PInternN P-CBCL internalizing <.0001 <.0001 < PExternN P-CBCL externalizing <.0001 <.0001 < j4ndx number of 5 DSM4 diagnoses <.0001 < j4ndep number of depressive symptoms <.0001 < j4ncd Number of DSM-IV CD symptoms <.0001 < j4nanx ANX: symptom scale score <.0001 <.0001 < j4nopp Number of DSM-IIIR ODD symptoms <.0001 <.0001 < p4nadh # of ADHD symptoms <.0001 <.0001 < j4nanx j4nopp p4nadh PTotalN P-CBCL total problems <.0001 <.0001 < pebdtot SDQ total difficulties scale <.0001 <.0001 < pemotion SDQ emotional symptoms scale adjusted for missing <.0001 <.0001 < pbehav SDQ joint total external difficulties score <.0001 <.0001 < PInternN P-CBCL internalizing <.0001 <.0001 < PExternN P-CBCL externalizing <.0001 <.0001 <.0001

20 j4ndx number of 5 DSM4 diagnoses <.0001 <.0001 < j4ndep number of depressive symptoms <.0001 <.0001 < j4ncd Number of DSM-IV CD symptoms <.0001 <.0001 < j4nanx ANX: symptom scale score <.0001 < j4nopp Number of DSM-IIIR ODD symptoms <.0001 < p4nadh # of ADHD symptoms <.0001 < CAPA 3 month summary statistics and correlations. Simple Statistics Variable N Mean Std Dev Sum Minimum Maximum PTotalN pebdtot pemotion pbehav PInternN PExternN j4ndx j4ndep j4ncd j4nanx j4nopp p4nadh Variable Label PTotalN P-CBCL total problems pebdtot SDQ total difficulties scale pemotion SDQ emotional symptoms scale adjusted for missing pbehav SDQ joint total external difficulties score PInternN P-CBCL internalizing PExternN P-CBCL externalizing j4ndx number of 5 DSM4 diagnoses j4ndep number of depressive symptoms j4ncd Number of DSM-IV CD symptoms j4nanx ANX: symptom scale score j4nopp Number of DSM-IIIR ODD symptoms p4nadh # of ADHD symptoms Pearson Correlation Coefficients Prob > r under H0: Rho=0 Number of Observations PTotalN pebdtot pemotion

21 PTotalN P-CBCL total problems <.0001 < pebdtot SDQ total difficulties scale <.0001 < pemotion SDQ emotional symptoms scale adjusted for missing <.0001 < pbehav SDQ joint total external difficulties score <.0001 <.0001 < PInternN P-CBCL internalizing <.0001 <.0001 < PExternN P-CBCL externalizing <.0001 <.0001 < j4ndx number of 5 DSM4 diagnoses <.0001 <.0001 < j4ndep number of depressive symptoms <.0001 <.0001 < j4ncd Number of DSM-IV CD symptoms <.0001 < j4nanx ANX: symptom scale score <.0001 <.0001 < j4nopp Number of DSM-IIIR ODD symptoms <.0001 <.0001 < p4nadh # of ADHD symptoms <.0001 <.0001 < pbehav PInternN PExternN PTotalN P-CBCL total problems <.0001 <.0001 < pebdtot SDQ total difficulties scale <.0001 <.0001 < pemotion SDQ emotional symptoms scale adjusted for missing <.0001 <.0001 < pbehav SDQ joint total external difficulties score <.0001 <

22 PInternN P-CBCL internalizing <.0001 < PExternN P-CBCL externalizing <.0001 < j4ndx number of 5 DSM4 diagnoses <.0001 <.0001 < j4ndep number of depressive symptoms <.0001 <.0001 < j4ncd Number of DSM-IV CD symptoms <.0001 <.0001 < j4nanx ANX: symptom scale score <.0001 <.0001 < j4nopp Number of DSM-IIIR ODD symptoms <.0001 <.0001 < p4nadh # of ADHD symptoms <.0001 <.0001 < j4ndx j4ndep j4ncd PTotalN P-CBCL total problems <.0001 <.0001 < pebdtot SDQ total difficulties scale <.0001 <.0001 < pemotion SDQ emotional symptoms scale adjusted for missing <.0001 < pbehav SDQ joint total external difficulties score <.0001 <.0001 < PInternN P-CBCL internalizing <.0001 <.0001 < PExternN P-CBCL externalizing <.0001 <.0001 < j4ndx number of 5 DSM4 diagnoses <.0001 < j4ndep number of depressive symptoms <.0001 <.0001

23 j4ncd Number of DSM-IV CD symptoms <.0001 < j4nanx ANX: symptom scale score <.0001 <.0001 < j4nopp Number of DSM-IIIR ODD symptoms <.0001 <.0001 < p4nadh # of ADHD symptoms <.0001 <.0001 < j4nanx j4nopp p4nadh PTotalN P-CBCL total problems <.0001 <.0001 < pebdtot SDQ total difficulties scale <.0001 <.0001 < pemotion SDQ emotional symptoms scale adjusted for missing <.0001 <.0001 < pbehav SDQ joint total external difficulties score <.0001 <.0001 < PInternN P-CBCL internalizing <.0001 <.0001 < PExternN P-CBCL externalizing <.0001 <.0001 < j4ndx number of 5 DSM4 diagnoses <.0001 <.0001 < j4ndep number of depressive symptoms <.0001 <.0001 < j4ncd Number of DSM-IV CD symptoms <.0001 <.0001 < j4nanx ANX: symptom scale score <.0001 < j4nopp Number of DSM-IIIR ODD symptoms <.0001 < p4nadh # of ADHD symptoms <.0001 <

24 Predicting any five DSM _IV diagnosis SCREENS Optimal Cut off Specificity Sensitivity PVP PVN Kappa 1M 3M 1M 3M 1M 3M 1M 3M 1M 3M 1M 3M INCAP SDQ_SUB SDQ_INC SDQJINN Legend: INCAP: Incapacity scale scores. SDQ_SUB = sum of five SDQ items 7, 8, 23, 13, 25. SDQ_INC=sum of SDQ items + j_inc, where j_inc is the binary incapacity diagnosis. SDQJINN=SDQ_SUB+INCAP SPECs and SENs under different cut offs for SED, ANY DEP and ANY DX diagnosis. DIAGNOSIS Serious Emotional Disorder (SED) Any Depression (ANY DEP) Any Diagnosis (ANY DX) Cut off % with SDQ cut off Specificity Sensitivity 1M 3M 1M 3M 1M 3M Prevalence of Diagnosis SED : 0.21 (CAPA 1M), 0.25 (CAPA 3M) ANY DEP: 0.10 (CAPA 1M), 0.11 (CAPA 3M) ANY DX: 0.28 (CAPA 1M), 0.36 (CAPA 3M) Notes: 1. Optimal cut off for ANY DX is 4 for both

25 CAPA1mo and CAPA3mo. 2. Optimal cut off for SED is 5 for CAPA1mo and 4 for CAPA3mo. 3. Optimal cut off for ANY DEP is 5 for both CAPA1mo and CAPA3mo. ROC 1. INCAPACITY SCALE VS ANY DX: CAPA 1 month ROC 2. INCAPACITY SCALE VS ANY DX: CAPA 3 month

26 ROC 3. SDQ_SUB VS ANY DX: CAPA 1 month

27 ROC 4. SDQ_SUB VS ANY DX: CAPA 3 month.

28 ROC 5. SDQ_INC VS ANY DX: CAPA 1 month

29 ROC 6. SDQ_INC VS ANY DX: CAPA 3 month.

30 ROC 7. SDQJINN VS ANY DX: CAPA 1 month.

31 ROC 8. SDQJINN VS ANY DX: CAPA 3 month.

32 APPENDIX 2 COMPARING CAPA AND DAWBA UNIGS SDQ and Sub-SDQ screens

33 Sample Sizes (parent-child pairs used in the analyses): Capa1 month: 362 pairs (C1) Capa3 month: 497 pairs (C3) Dawba: 636 pairs (D) COMPARING CAPA and DAWBA USING SDQ and SDQ_SUB SCREEN: ANY DSM-IV DIAGNOSIS Optimal SCREEN Screen Sp Se PVP PVN Kappa Cut Point C1 C3 D C1 C3 D C1 C3 D C1 C3 D C1 C3 D C1 C3 D SDQ SDQ_SUB Prevalence of Any diagnosis: 27.90% (Capa 1 month, # cases = 101) 32.4% (Capa 3 month, # cases = 161), 17.77% (DAWBA, # cases =113 ). Proportions screen hi using the best cut-off for ANY DX: CAPA 1 Month: P(SDQ_opt) = (# cases=98) P(SDQ_SUB_opt) = (# cases=87) CAPA 3 Month: P(SDQ_opt) = (#cases=174) P(SDQ_SUB_opt) = (# cases=193) DAWBA: P(SDQ_opt) = (# cases= 91) P(SDQ_SUB_opt) = (# cases= 124) SDQ and SDQ_SUB on SED using Capa 1 Month and Capa 3 Month SED diagnoses, and Dawba EB diagnosis Optimal SCREEN Screen Sp Se PVP PVN Kappa Cut Point C1 C3 D C1 C3 D C1 C3 D C1 C3 D C1 C3 D C1 C3 D SDQ SDQ_SUB Prevalence of SED: 28.73% (Capa 1 month, #cases=104) 30.99% (Capa 3 month, #cases=154)

34 16.82% (Dawba, #cases = 107) Proportions screen hi using the best cut-off for SED: CAPA 1 Month: P(SDQ_opt) = (# cases=98) P(SDQ_SUB_opt) = (# cases=87) CAPA 3 Month: P(SDQ_opt) = (#cases=174) P(SDQ_SUB_opt) = (# cases=99) DAWBA: P(SDQ_opt) = (# cases= 114) P(SDQ_SUB_opt) = (# cases= 61)