A Cross-sectional, Randomized, Non-interventional Methods Study to Compare Three Methods of Assessing Suicidality in Psychiatric Inpatients

Transcription

1 A Cross-sectional, Randomized, Non-interventional Methods Study to Compare Three Methods of Assessing Suicidality in Psychiatric Inpatients Eric A. Youngstrom, Ph.D., Ahmad Hameed, M.D., Michael Mitchell, B.A., Andrew J. Freeman, M.A., Anna Van Meter, M.A., Guillermo Perez Algorta, Ph.D., Alan Gelenberg, M.D., & Roger E. Meyer, M.D.

2 Acknowledgments Study supported by an investigator initiated award from Pfizer, Inc. to Penn State Hershey Medical Center Alan Gelenberg, MD PI

3 Primary objective: Compare self-report and clinical interviewbased versions of the older and current editions of the Sheehan Suicide Tracking Scale (S-STS) to the C-SSRS in the assessment of suicidal ideation/behavior in the context of the domains defined by the C-CASA and in relationship to the clinical evaluation of recently admitted psychiatric inpatients.

4 "'In this world nothing can be said to be certain, except death and taxes." -Benjamin Franklin, 1789 I am realizing Franklin was not a clinical trialist.

5 C-CASA Classification (circa 2010) Item Classification Mapping 1 Completed suicide Impossible to interview 2 Suicide Attempt CCASA #2 3 Preparatory suicidal behavior CCASA #3 4 Suicidal Ideation CCASA #4 5 Other, no deliberate self harm NA in CSSRS interview 6 Fatal, but not enough information to classify Impossible to interview 7 Self-injurious act, no suicidal intent (NSSI) CCASA #7 8 Self-injurious behavior, intention unknown Interview should not have unknown 9 Nonfatal, not enough information to classify

6 C-CASA Classification (August 2012)

7 Secondary Objectives Quantify relative ease, time and means of administration (self-report and clinical interview) Inter-rater and Intra-rater reliability (over time) of the scoring of rater administered form Patient preference re: acceptability of the instruments

8 Urn Randomization Stratified variables: Age (up to 24 years versus 25 and older) Psychosis (present/absent) Two arms: C-SSRS, new S-STS self report, old S-STS interview C-SSRS, old S-STS self report, new S-STS interview Order counterbalanced within each arm 6 sequences

9 Other design features Break between interviews Same day completion whenever possible Same interviewer Video recording for re-rating by different judge (n = 90 will be re-rated) Raters trained by Posner and by Sheehan

10 Demographics (N =199) Variable Descriptive Age M = 38.5 (SD = 12.4) 18 to 65 years Gender 113 female (57%) Race 157 White (79%), 37 Black (18%), 9 American Indian (5%), 6 Asian/Pacific (3%) Hispanic 26 (13%) Married 40 (20%) Education 87% with GED or higher Raters 3 interviewers, 3 different video raters Videos 60 rated (more than 90 available)

11 Administration Times C-SSRS > New S-STS > Old STS all p <.001 Five minutes or less (ecssrs sig faster)

12 Minutes to Administer C-SSRS Old STS New S-STS Minimum th Pct Median th Pct Maximum Mean (SD) 17.8 (8.7) 6.5 (2.7) 16.2 (7.4) N = 199

13 Interview Length Much Longer if Endorse Suicidal Ideation

14 What is a good kappa? K Excellent Good Fair Poor Cicchetti & Sparrow, 1981 Excellent Fair to Good Poor Fleiss, 1981 Almost Perfect Substantial Moderate Fair Slight Poor Landis & Koch (1977) (Excellent) Very Good Good Questionable Unacceptable Regier et al. (2012) DSM-5

15 Reliability Standoff Two sources of error affect each estimate: Reliability of Test A, Reliability of Test B C-SSRS Kappa Kappa New S-STS Kappa Old STS

16 Kappa for Lifetime Attempts (C-CASA #2).68 (.06) C-SSRS.66 (.06) New S-STS.68 (.06) Old STS

17 Recommended approach for missing or imperfect gold standard Pepe, M. S. (2003). The statistical evaluation of medical tests for classification and prediction. New York: Wiley. Zhou, X.-H., Obuchowski, N. A., & McClish, D. K. (2002). Statistical methods in diagnostic medicine. New York: Wiley.

18 LCA Approach Put all converging measures into analysis Fit two class solution: Diagnosis+ vs. Diagnosis-, Haves vs. Have Nots Save predicted class membership Can estimate sensitivity and specificity of each measure calibrated against latent class (could also test larger number of classes, but diagnostic efficiency statistics no longer meaningful)

19 Latent Class Model True status, cannot measure perfectly Latent Class Drives performance on measured variables C-SSRS New S-STS Old STS

20 Latent Class Model Suicidal Behavior (C-CASA 2) Lifetime Kappa Sensitivity Specificity C-SSRS New S-STS Old STS C-CASA 2 Behavior None perfect, all excellent C-SSRS New S-STS Old STS

21 Latent Class Model Preparatory Acts (C-CASA 3) Lifetime C-CASA 3 Prep acts Kappa C-SSRS.86 New S-STS.73 Old STS.63 Weakest agreement across the measures C-SSRS New S-STS Old STS

22 Latent Class Model Ideation (C-CASA 4) Lifetime Kappa Sensitivity Specificity C-SSRS New S-STS Old STS C-CASA 4 Ideation C-SSRS New S-STS Old STS

23 Latent Class Model NSSI (C-CASA 7) Lifetime C-CASA 7 NSSI Kappa C-SSRS.78 New S-STS.77 Old STS.83 C-SSRS New S-STS Old STS

24 Kappas for Lifetime Ratings K.6.5 New S-STS vs. C-SSRS Old S-STS vs. C-SSRS.4 New S-STS vs. Old S-STS CASA-2 Attempts CASA-3 Prep CASA-4 Ideation CASA-7 NSSI

25 Kappas for Lifetime Ratings K New S-STS vs. C-SSRS Old S-STS vs. C-SSRS New S-STS vs. Old S-STS New S-STS vs. LCA CSSRS vs. LCA Old S-STS vs. LCA CASA-2 Attempts CASA-3 Prep CASA-4 Ideation CASA-7 NSSI

26 Kappas for Lifetime Ratings Excellent K Good Fair.4.3 Poor Attempts CASA-2 Prep CASA-3 Ideation CASA-4 NSSI CASA-7 New S-STS vs. LCA CSSRS vs. LCA Old S-STS vs. LCA

27 Kappas for Lifetime Ratings across 4 C-CASA categories K New S-STS vs. C-SSRS Old S-STS vs. C-SSRS New S-STS vs. Old S-STS New S-STS vs. LCA CSSRS vs. LCA Old S-STS vs. LCA Attempts CASA-2 Prep CASA-3 Ideation CASA-4 NSSI CASA-7

28 Summary Suicidal Behavior Past Month N=73 (37%) Kappa BCa Low High Sensitivity Specificity NPV PPV CSSRS New S-STS Old S-STS Lifetime N=152 (76%) CSSRS New S-STS Old S-STS Median All values sig p <.0005

29 Summary Suicidal Ideation Past Month N=180(90%) Kappa BCa Low High Sensitivity Specificity NPV PPV CSSRS New S-STS Old S-STS Lifetime N=195(98%) CSSRS New S-STS Old S-STS Median All values sig p <.0005

30 Behavior: Kappa and 95% c.i. (BCa) K Past Month Lifetime

31 Ideation: Kappa and 95% c.i. (BCa) K Past Month Lifetime

32 Inter-Rater Reliability (more tape coding planned a priori) Lifetime C-SSRS (N =60) New S-STS (n =27) Old STS (n =33) CASA #2 (Attempt).893 (.060) < (.181) (.098) <.0005 CASA #3 (Prep).651 (.106) < (.178) ns.857 (.097) <.0005 CASA #4 (Ideation).551 (.226) < a -- a CASA #7 (NSSI).799 (.078) < (.137) < (.106) <.0005 Past Month CASA #2 (Attempt).720 (.097) < (.144) < (.062) <.0005 CASA #3 (Prep).800 (.077) < (.163) (.082) <.0005 CASA #4 (Ideation) (.000) < (.129) < (.000) <.0005 CASA #7 (NSSI).795 (.114) < (.131) < (.155) <.0005 a Cannot estimate % ideation on cases rated

33 Preliminary Inter-Rater Reliability CASA 2012 Categories C-SSRS (N =60) CASA 2012 Category C-SSRS Lifetime C-SSRS Past Month #1 Passive Ideation.734 (.178).924 (.075) #2 Ideation (no method, intent, plan).703 (.160).957 (.043) #3 Ideation+Method (no intent, plan).774 (.124).926 (.052) #4 Ideation+Method+Intent (no plan).884 (.065).934 (.046) #5 Ideation+Method+Intent +Plan.770 (.081).688 (.093) #7 Attempt.893 (.060).720 (.097) #8 Interrupted Attempt.678 (.103).573 (.137) #9 Aborted Attempt.602 (.101).946 (.053) #10 Prep Acts.761 (.085).804 (.094) #11 NSSI.799 (.078).795 (.114) No completed suicides (CASA 2012 #6); all K significant p <.0005

34 Patient Acceptance: Majority highly satisfied d =.08 d =.40 Patient Acceptance (% Max) CSSRS New S-STS CSSRS Old STS

35 Key Points Validity of all three versions excellent Using better model yields more accurate estimate Agreement with each other good Inter-rater reliability (video) excellent Similar patient, interviewer preferences & tolerability Multiple endpoints complicates analyses and interpretation

36 Limitations Did not compare CSSRS interview to E version Inpatient unit meant extremely high rate of ideation Important clinical setting & context Challenging for tests (big standard errors) Consumer Reports model generalization across clinical settings vital Need to explore moderators of accuracy Did not assess prospective prediction or treatment response

37 Future Directions Investigate where the signal is Brown s talk severe ideation more important than mild Start with simple models and measures Have data guide decisions to add complexity

38 Complexity has a cost Categories df Type I Error Rate Evidence about Model Suicidal? Yes/No 1.05 Probably oversimplified Ideation, Behavior 2.10 (Could differentiate severe ideation from mild ) C-CASA 2010, Interview 4.19 C-CASA May be too complex Broadly read C-CASA Definitely too complex Endpoints probably not all equally important clinically Cannot easily adjust Type II error too serious What are costs of false positive in drug development or post marketing?

39 "Everything should be made as simple as possible, but not simpler." -Albert Einstein, 1933

40 Thank You!

41 Kappa for Lifetime Preparatory Acts (C-CASA #3).59 (.06) C-SSRS.47 (.07) New S-STS.39 (.06) Old STS

42 Kappa for Lifetime Ideation (C-CASA #4).66 (.19) C-SSRS.60 (.19) New S-STS.76 (.13) Old STS

43 Kappa for Lifetime NSSI (C-CASA #7).56 (.06) C-SSRS.61 (.05) New S-STS.61 (.06) Old STS

44