Study Data Excluded Reason for Exclusions

Transcription

1 Data Supplement for Sharf, J., Primavera, L.H., and Diener, M. J. (2010). Dropout and Therapeutic Alliance: A Meta-Analysis of Adult Individual Psychotherapy, Psychotherapy Theory, Research, Practice, Training. Supplementary Table 1 Data Excluded from the Meta-analysis of Alliance and Psychotherapy Dropout Study Data Excluded Reason for Exclusions Frank & Gunderson (1990) Data found in Table 3 (p. 232). Correlation between Active Engagement ratings at 6 months and length of stay in psychotherapy, r (129) = -.42, p <.001 (note that the negative correlation is actually in the predicted direction since lower Active Engagement scores actually indicate higher engagement [see p. 230]). Partial correlations between Active Engagement ratings at 6 months and length of stay in psychotherapy, controlled for length of initial hospitalizations, for baseline levels of cognitive disorganization and ego weakness, premorbid social and vocational adjustment, social class, amount of psychotherapy previously received, and level of functioning at 6 months. This table does not provide actual alliance data for dropouts but instead assigns them a value of none automatically based on the fact that they dropped out of treatment. Given that the dependent variable is length of stay in psychotherapy (rather than a dichotomous categorization of completers versus dropouts ), these data do not fit with the present meta-analysis s eligibility criteria. Data generated using multivariate analysis cannot generally be included in a metaanalysis (Lipsey & Wilson, 2001).

2 Meta-Analysis of Dropout and Alliance, Data Supplement 2 Omnibus comparison between dropouts and More specific, focused contrast data has completers across several levels of alliance already been included in the present metaanalysis. ratings, χ 2 (4) = 26.76, p <.001. Kokotovic & Tracey (1990) Comparisons between dropouts and completers in terms of Working Alliance Inventory (WAI) Bond and Task scales for both client and counselor ratings. Based on reliability data reported in Kokotovic and Tracey (1990, p. 18), the Goal scale of the WAI demonstrated the highest reliability; thus, only data for the Goal scale was included, whereas data for the Task and Bond scales were excluded. Mohl et al. (1991) Comparisons between dropouts and completers in terms of specific items on the Helping Alliance scale. The comparison between dropouts and completers on average scores across the entire Helping Alliance scale were already included in the present meta-analysis. Comparisons between dropouts and completers Only the most reliable and valid measure was in terms of Osgood Semantic Differential scores selected when multiple measures of a single on several stimuli, including, The doctor who construct were reported in an individual study. interviewed me was, and I expect Therefore, only data from the Helping Alliance psychotherapy to be. scale was included, whereas these additional data were excluded. Piper et al. (1999) The data comparing dropouts versus completers in terms of early session(s) alliance ratings. The decision of whether to include early versus late alliance ratings was not clear-cut. Martin, Garske, and Davis (2000) reported in their meta-analytic review that disaggregating effect sizes measuring the relationship between alliance and outcome based on time of alliance rating did not account for additional variance

3 Meta-Analysis of Dropout and Alliance, Data Supplement 3 in the model. Horvath and Symonds (1991) did report a slight superiority of the predictive value of early (generally defined as first to fifth session) alliance ratings (mean r =.31) compared to that of late (at or near the end of therapy) alliance ratings (mean r =.30) in their meta-analysis, although they concluded that the values for early and late alliance are nearly identical (p. 145). Taken together with the fact that some of the specific early session(s) data (i.e., the comparison between completers and dropouts in terms of the therapist ratings) in Piper et al. (1999) were reported simply as nonsignificant (which would have meant coding these results conservatively as an effect size of zero), it was decided to include only the data from the last session alliance ratings (which consisted of therapist ratings only) and exclude the early session(s) alliance ratings (which included both client and therapist ratings). Data comparing completers versus dropouts on the Vanderbilt Psychotherapy Process Scale (VPPS). The VPPS does not specifically measure alliance. Martin et al. (2000) note, Although the VPPS measured features of the therapistclient relationship, it was not specifically designed to measure the alliance. (p. 439)

4 Meta-Analysis of Dropout and Alliance, Data Supplement 4 Samstag et al. (1998) Data comparing completers with good outcome versus dropouts in terms of specific scale ratings on the WAI short version (WAI-S). Data comparing completers with good outcome versus dropouts in terms of the WAI- S Total scale were already included in the present meta-analysis. Startup et al. (2006) Comparison between dropouts versus completers in terms of ratings on the Active Engagement (AE) scale. Data presented in Startup et al. (2006) suggest superior reliability for the WAI relative to the AE scale. Thus, only data for the WAI were included in the present meta-analysis, whereas data for the AE scale were excluded. Comparisons between dropouts versus Data presented in Startup et al. (2006) suggest completers in terms of ratings on the Goal and superior reliability for the WAI Bond subscale Task subscales of the WAI. relative to the Goal and Task subscales. Thus, only data for the WAI Bond subscale were included in the present meta-analysis, whereas data for the Goal and Task subscales of the WAI were excluded. Tryon & Kane (1990) Comparison between dropouts and completers in terms of the following Helping Alliance Questionnaire (HAQ) subscales: Counselor Helpfulness and Client-counselor Bond. Data comparing dropouts and completers on the total score of the HAQ were already included in the present meta-analysis.

5 Meta-Analysis of Dropout and Alliance, Data Supplement 5 Supplementary Table 2 Moderator Codings Study Name % Male % H.S. or % Treatment Age Primary Setting Dropout Definition more Caucasian Length Diagnosis Arnow et al. (2007) Medium Mood D/O Research clinic Failed to complete treatment Frank & Gunderson Psychotic Inpatient (1990) D/O Johansson & Eklund (2006) Mixed Outpatient # sessions early (2-5, 1 month) Kokotovic & Tracey Mixed Counseling Mixed other (1990) center Mohl et al. (1991) Outpatient # sessions early (2-5, 1 month) Piper et al. (1999) Longer Mixed Outpatient # sessions middle (6-20, 2-5 months) Samstag et al. (1998) Longer Mixed Research clinic Combined therapist judgment and # sessions Startup et al. (2006) 0.76 Longer Psychotic Inpatient # sessions middle (6-20, 2-5 months) Tryon & Kane (1990) Mixed Counseling center Failure to attend scheduled session Tryon & Kane (1993) Medium Mixed Counseling Failure to attend center Tryon & Kane (1995) Longer Mixed Counseling center Note. H.S. = High school; = Study did not provide this data; D/O = Disorder. scheduled session Failure to attend scheduled session

6 Meta-Analysis of Dropout and Alliance, Data Supplement 6 Supplementary Table 3 Overall Random Effects Meta-Analysis of Alliance and Dropout 95% confidence interval Study Name d Lower Limit Upper Limit Z-Value p-value Arnow et al. (2007) Frank & Gunderson (1990) <0.001 Johansson & Eklund (2006) Kokotovic & Tracey (1990) Mohl et al. (1991) Piper et al. (1999) Samstag et al. (1998) <0.001 Startup et al. (2006) Tryon & Kane (1990) Tryon & Kane (1993) Tryon & Kane (1995) Weighted Mean Effect Size <0.001 Note. d refers to the standardized mean difference score defined in the method section of the article to which this data supplement is attached.

7 Meta-Analysis of Dropout and Alliance, Data Supplement 7 Supplementary Table 4 Subgroup Random Effects Meta-Analytic Results Calculated by Treatment Setting Weighted Mean Effect Size Treatment Setting d 95% CI Z p-value Q df p- value Inpatient a Outpatient b Heterogeneity I 2 T 2 T 4.97 < < Research clinic c < Counseling center d Note. d refers to the standardized mean difference score defined in the method section of the article to which this data supplement is attached. I 2 refers to the proportion of observed variance that can be attributed to real differences in effect size (Borenstein, Hedges, Higgins, & Rothstein, 2009). T 2 refers to the estimate of the variance in true effect sizes (Borenstein et al. 2009). T refers to the estimate of the standard deviation of the true effect sizes (Borenstein et al., 2009). a k = 2. b k = 3. c k = 2. d k = 4.

8 Meta-Analysis of Dropout and Alliance, Data Supplement 8 Supplementary Table 5 Subgroup Random Effects Meta-Analytic Results Calculated by Alliance Rater Weighted Mean Effect Size Alliance Rater d 95% CI Z p-value Q df p- value Heterogeneity I 2 T 2 T All a < Therapist/Staff/Observer b < Therapist/Staff c < Patient c < Therapist d < Note. All refers to ratings provided by any combination of the following individuals: patients, therapists, staff members, or observers. d refers to the standardized mean difference score defined above in the method section. I 2 refers to the proportion of observed variance that can be attributed to real differences in effect size (Borenstein et al. 2009). T 2 refers to the estimate of the variance in true effect sizes (Borenstein et al. 2009). T refers to the estimate of the standard deviation of the true effect sizes (Borenstein et al., 2009). a k = 11. b k = 9. c k = 8. d k = 7.

9 Meta-Analysis of Dropout and Alliance, Data Supplement 9 Supplementary Table 6 Mixed effects regression model (method of moments) regression results for education (k = 6) Point estimate Mixed effects, Z-distribution Standard error 95% Lower 95% Upper Z-value Intercept p-value Education

10 Meta-Analysis of Dropout and Alliance, Data Supplement 10 Supplementary Table 7 Mixed effects regression model (method of moments) test of the model for education Test of the model: Simultaneous test that all coefficients (excluding intercept) are zero Q model = 4.00, df = 1, p =.045 Goodness of fit: Test that unexplained variance is zero T 2 =.003, Q residual = 4.04, df = 4, p =.40

11 Meta-Analysis of Dropout and Alliance, Data Supplement 11 Supplementary Table 8 Mixed effects regression model (method of moments) regression results for treatment length (k = 6) Point estimate Mixed effects, Z-distribution Standard error 95% Lower 95% Upper Z-value Intercept Treatment Length p-value

12 Meta-Analysis of Dropout and Alliance, Data Supplement 12 Supplementary Table 9 Mixed effects regression model (method of moments) test of the model for treatment length Test of the model: Simultaneous test that all coefficients (excluding intercept) are zero Q model = 5.93, df = 1, p =.01 Goodness of fit: Test that unexplained variance is zero T 2 =.00, Q residual = 3.21, df = 4, p =.52

13 Meta-Analysis of Dropout and Alliance, Data Supplement 13 References * References marked with a single asterisk indicate studies which were included in the metaanalysis. * Arnow, B., Blasey, C., Manber, R., Constantino, M., Markowitz, J., Klein, D., Rush, A. J. (2007). Dropouts versus completers among chronically depressed outpatients. Journal of Affective Disorders, 97(1-3), Borenstein, M., Hedges, L.V., Higgins, J.P.T., & Rothstein. H.R. (2009). Introduction to metaanalysis. Chichester, UK: John Wiley & Sons, Ltd. *Frank, A. F., & Gunderson, J. G. (1990). The role of the therapeutic alliance in the treatment of schizophrenia: Relationship to course and outcome. Archives of General Psychiatry, 47(3), *Johansson, H., & Eklund, M. (2006). Helping alliance and early dropout from psychiatric outpatient care: The influence of patient factors. Social Psychiatry and Psychiatric Epidemiology, 41(2), *Kokotovic, A. M., & Tracey, T. J. (1990). Working alliance in the early phase of counseling. Journal of Counseling Psychology, 37(1), Lipsey, M.W., & Wilson, D.B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage Publications. Martin, D., Garske, J., & Davis, M. (2000). Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review. Journal of Consulting and Clinical Psychology, 68, *Mohl, P. C., Martinez, D., Ticknor, C., & Huang, M. (1991). Early dropouts from psychotherapy. Journal of Nervous and Mental Disease, 179(8), *Piper, W. E., Ogrodniczuk, J. S., Joyce, A. S., McCallum, M., Rosie, J. S., O Kelly, J. G., & Steinberg, P. I. (1999). Prediction of dropping out in time-limited, interpretive individual psychotherapy. Psychotherapy: Theory, Research, Practice, Training, 36(2), *Samstag, L. W., Batchelder, S. T., Muran, J. C., Safran, J. D., & Winston, A. (1998). Early identification of treatment failures in short-term psychotherapy: An assessment of therapeutic alliance and interpersonal behavior. Journal of Psychotherapy Practice & Research, 7(2), *Startup, M., Wilding, N., & Startup, S. (2006). Patient treatment adherence in cognitive behaviour therapy for acute psychosis: The role of recovery style and working alliance. Behavioural and Cognitive Psychotherapy, 34(2), *Tryon, G. S., & Kane, A. S. (1990). The helping alliance and premature termination. Counselling Psychology Quarterly, 3(3), *Tryon, G. S., & Kane, A. S. (1993). Relationship of working alliance to mutual and unilateral termination. Journal of Counseling Psychology, 40(1), *Tryon, G. S., & Kane, A. S. (1995). Client involvement, working alliance, and type of therapy termination. Psychotherapy Research, 5(3),