Roles of Common Factors & Therapist Effects in Therapy Outcomes. Session #0931. G.S. (Jeb) Brown, Ph.D. Center for Clinical Informatics
|
|
- Gabriella Payne
- 6 years ago
- Views:
Transcription
1 Roles of Common Factors & Therapist Effects in Therapy Outcomes Session #0931 G.S. (Jeb) Brown, Ph.D. Center for Clinical Informatics
2 Common factors The effectiveness of all treatments is due, in some part, to factors common to all treatments. Contact with a helping, caring professional fosters hope and expectancy. We have come to accept the potency of placebo effects, and insist that the effectiveness bona fide treatments exceeds that of placebo treatments So far, so good. Who can argue with this?
3 Randomized double-blind placebo controlled drug trials Double blind placebo controlled drug studies provide an exemplar for estimating the role of common factors. Traditionally, the drug is interpreted as the difference between placebo and the active drug. Meta-analysis of multiple studies of antidepressants lets us estimate the relative importance of common factors (placebo effects) versus drug effects.
4 Meta-analyses and placebo Meta-analysis involves the use of a statistical techniques to combine results from multiple studies in order in an effort to generalize findings. Meta-analysis of multiple studies of antidepressants let us estimate the relative importance of common factors (placebo effects) versus drug effects. 1-3
5 Drug effect accounted for 25% of measured improvement Antidepressant effect (25%) Placebo effect Natural course of the illness
6 Evidenced based psychotherapy For several decades psychotherapy researchers have attempted to design randomly controlled trails (RCT) to investigate the effectiveness of specific methods of psychotherapy. Study design analogous to pharmacy trials, except that designing credible placebo treatments is much more problematic. Various treatment methods are being touted as evidenced based by citing the number of RCTs providing evidence that the treatment exceeded placebo (or some other treatment).
7 Psychotherapy brands The advocacy for the use of specific therapies is analogous to the advertising of brands of antidepressant medication. Calls for wide spread use of evidence based treatments in psychotherapy is analogous to the FDA s insistence that a drug may not be marketed for the treatment of depression until at least two studies have shown superiority to placebo. Advocates and practitioners of various evidence based treatments have a vested interest in discouraging the use of unproven treatments.
8 Brand differentiation Advocates of psychotherapy brands insist on the uniqueness of their therapy and the need to adhere to specific treatment procedures Research methodology requires the use of manuals and other techniques to standardize treatments Treatment effectiveness presumed to be dependent on the correct application of the active ingredients in the psychotherapy method.
9 The Dodo Bird Effect Rosenzweig S. (1936) Some implicit common factors in diverse methods of psychotherapy: At last the Dodo said, Everybody has won and all must have prizes. Am J Orthopsychiatry 6:412-5.
10 The Dodo Bird Lives! Wampold BE, Mondin GW, Moody M, et al. (1997). A meta-analysis of outcome studies comparing bona fide psychotherapies: Empirically, All must have prizes. Psychol Bull 122: Luborsky, L., Rosenthal, R., Diguer, L., et al The dodo bird verdict is alive and well--mostly. J. Psychotherapy Integration Vol 12(1) 32-57
11 Meta-analysis & common factors Over two decades of meta-analytic studies have served to reinforce Rosenzweig s 1936 observation that different methods of psychotherapy tend to produce comparable outcomes the Dodo Bird Effect Lack of evidence for specific treatment effects bolster the argument that almost all of the effects of psychotherapy are due to factors common to all psychotherapies. 5-11
12 Real world example Human Affairs International (HAI) collected outcome data from a large number of clinicians between 1996 and Clinicians were asked to specify the primary method of psychotherapy (or medication management only) Analyses revealed no significant differences in the outcome or mean number of sessions across all treatment methods, including medication management.
13 Treatment & outcome HAI data 0.40 Mean Value of Residual Change after Controlling for Severity (positive values = greater than average change; 90% confidence interval) Residual Effect Size Insight Oriented (n=135) Solution Focused (n=647) Behavioral (n=67) Cognitive Behavioral (n=1226) Fam ily (n=152) Medication Management (n=514) Other (n=129) Unknown (n=85) -0.20
14 Fidelity and practicality Propagation of evidenced based treatment methods requires some method of measuring fidelity to the treatment. BIG PROBLEM! If clinicians reports using Cognitive Behavioral Therapy, how can we have any confidence that what that clinician did with a specific patient was comparable to the Cognitive Behavioral Therapy in the RCTs?
15 Goldilocks Effect Clinicians tend to be eclectic and flexible in their choice of treatment methods. Clinicians and patients tend to try different treatments until they find something that is just about right for them. Patients and clinicians tend to adjust the number and frequency of sessions depending on the patient s level of distress and rate of improvement. Result: All treatments appear to have similar outcomes and patients with good outcomes tend to use fewer session than patients with poor outcomes.
16 Recommended reading Rigorous review and analyses of controlled studies on psychotherapy outcome. Conclusion: much more variance resides with the clinician than with the treatments.
17 Therapists effects Wampold and others argue that researchers have ignored the individual therapist as a source of variance. 11, The person of the therapist is necessary to delivery the treatment, and personal characteristics of the therapist modify the effect of the treatment. Factors contributing to therapists effects may include elements clinical skill and knowledge as well as personality traits.
18 RCT and ANOVA brief history Some of the earliest applications of randomized control group design and analysis of variance were in agriculture and education. 12,13 RCT methodology later adopted by medicine and eventually psychotherapy research. 11,14 Simple ANOVA is appropriate only if the individual farmer, teacher or clinician has little or no impact on the effectiveness of the farming, teaching or treatment method!
19 HLM & therapist effects Hierarchical Linear Modeling (HLM) is an advance in statistical methodology that permits us to model variance at the clinicians level and as well as the treatment level. An rapidly growing body of published research points to the conclusion that therapist effects almost certainly exceed specific treatment effects by a large margin.
20 Variance due to the clinician Published research making use of HLM points to the conclusion that the clinician accounts for much more of the variance in psychotherapy outcomes that treatment method per se. 11, Analyses of PacifiCare Behavioral Health s massive database database on patient outcomes confirms significant variance in psychotherapy outcomes at the clinician level. 24,25
21 PacifiCare Behavioral Health ALERT System Initiated an outcomes management program in 1998 using 30 item patient self report questionnaires administered at regular intervals in treatment. ALERT System used to capture data and monitor patient outcomes in real time. Currently over 7,000 clinicians are contributing outcome data on a regular basis. Probably largest database on mental health outcomes in the world.
22 PBH research collaboration PBH actively sought the involvement of leading psychotherapy outcomes researchers from leading academic institutions. External researchers actively involved in design of the measurement system and ongoing analysis of the data. PBH encouraged publication of findings in academic journals.
23 The (almost) Bell Curve PBH data Solo clinicians with sample sizes => 20 25% % of clinicians 20% 15% 10% 5% 0% Effect Size
24 Where is the variance? Unexplained 74% (external stressors, health condition, social supports, etc.) Acuity (intake score) 18% Diagnosis, age & sex 2% Clinicians & treatments Clinician 6% Treatment method 1%
25 % of variance due to therapists in the real world Analysis of PacifiCare Behavioral Health (PBH) data reveals 6% of variance due to therapist. 25 Patients on medication have a higher % of variance due the therapist than those receiving psychotherapy alone. Huh??
26 Therapists and meds Outcomes (residualized scores) of 15 therapists for patients with concurrent medication or no medication meds nom eds
27 Test scores and medication PBH data Percentage of cases on medication as a function of intake score 80% 70% 60% 50% 40% 30% 20% 10% 0%
28 Which treatment is best? Goldilocks Effect: Clients tend to get the treatment that is just about right for them. Treatment, severity and outcome 2.00 Effect size Psychotherapy only Psychotherapy and medication Normal functioning Intake score Severe symptoms
29 Cross validation analysis Psychotherapists in PBH network ranked based on all cases from if sample size =>30; N=116. If a therapist s mean residualized final score < 0 then clinician rated Highly effective ; else clinician rated Less effective. Outcomes evaluated in the cross validation period for a new sample of cases.
30 Cross validation results Effect size Clinician cross validation results to 2004 Therapists assessed on at least 30 cases between Highly effective clinicians had mean Change Index Score > Intake scores (mean split) Highly effective clinicianspsychotherapy only Highly effective clinicianspsychotherapy and medication Less effective clinicianspsychotherapy only Less effective clinicianspsychotherapy and medication
31 Risk of not using HLM Wampold and colleagues at the University of Wisconsin recently reanalyzed data from the National Institute of Mental Health s Treatment of Depression Collaborative Research Program (TDCRP) study using HLM Prior published reports found significant differences between two methods of psychotherapy as well as between placebo and antidepressant medication. Reanalysis of psychotherapy data using HLM revealed that 0% of the variance was due to the psychotherapy methods, while 8% was attributable to the therapists. 27
32 Psychiatrist effects Wampold and colleagues also used HLM to reanalyze the results antidepressant and placebo legs of the TDCRP study. 28 Included the 9 individual psychiatrists as a variable. Outcome measured by change on patient self report measure (Beck Depression Inventory). 9.1% of the variance due to the psychiatrist; only 3.4% due to the medication. Top third of psychiatrists achieved a better outcome with placebo than bottom third achieved with the antidepressant.
33 Placebo & therapist effects Hypothesis: Placebo/common factor effects are mediated by the clinician/patient relationship. Common factors tend to account for much more of the variance than specific treatment effects. If the effects of common factors are mediated by the clinician/patient relationship, then we would naturally find much of the variance in outcomes would be due to the clinician. The human factor matters! DUH!
34 What s a clinician to do? If a wide variety of treatments appear to be equally efficacious, can a therapist do to achieve the best outcomes possible for their patients? A growing body of research supports the use of repeated administrations of patient self report outcome questionnaires to monitor response to treatment. Routine measurement and and early identification of patients with a poor response to treatment has been shown to reduce treatment failures
35 Therapeutic alliance A large body of evidence suggests that the relationship and working alliance between the clinicians and patient is an important factor in the outcome Routine use of a session rating/therapeutic alliance scale may permit clinicians to identify and repair problems in the working alliance.
36 Outcomes informed care Meta-method designed to improve outcomes across all patients and diagnoses, regardless of treatment method. Routine use of patient self report questionnaires to track symptom severity and therapeutic alliance. Use of feedback mechanisms to alert clinicians to patients at risk for poor outcomes. Performance feedback to clinicians, including comparison to outcomes to those of clinicians treating similar patients. Preferential referrals to highly effective clinicians.
37 2 case studies Resources for Living (RFL) provides telephonic EAP services. Customers include Walmart. Accountable Behavioral Healthcare Alliance (ABHA) is a managed behavioral healthcare organization servicing Oregon Health Plan members in 5 rural county area
38 Case history # 1: RFL Began using the 4 item Outcome Rating and Session Rating Scales in 2002 Administered over the phone as part of telephonic counseling sessions. Baseline data collected for 5 months Baseline data used to create trajectory of change graphs Real time feedback provided to counselors via SIGNAL System
39 RFL Signal System: results Training and feedback Effect size Baseline period 2nd quarter rd quarter th quarter st quarter nd quarter rd quarter th quarter st quarter nd quarter rd quarter 2004
40 Case history # 2: ABHA Began utilizing the 4 item Oregon Change Index (OCI) in OCI administered at every session in outpatient and day treatment settings. OCIs collected at over 80% of all sessions. Collected baseline data for 18 months, began giving feedback in mid Updated Excel based Active Case Report contains outcome data on all cases seen within the last 6 weeks is ed to the clinicians weekly.
41 ABHA results ABHA Effect Size by Quarter Clients with scores in clinical range at intake Effect SIze Benchmark from clinical trials:.83 1st 04 2nd 04 3rd 04 4th 04 1st 05 2nd 05 3rd 05 4th 05 1st 06
42 Implications for clinicians Good news: The clinician matters!!!!!! All treatments (including medications!?) are only as effective as the clinicians delivering the treatment. Clinicians have an ethical responsibility to assess and improve their personal effectiveness as clinicians they cannot rely on the treatments alone to be curative. Effective clinicians deliver high value services and are worth more money!!!
43 Implications for administrators & policy makers Exclusive focus on the effectiveness of treatments rather than the value of the clinicians limits the potential to improve outcomes. Use of effective clinicians tends to lower costs. Administrators and policy makers have an obligation to consumers to assure that they have access to effective clinicians. Failure to monitor outcomes at the clinician level places consumers at risk.
44 References 1. Kirsch, I & Sapirstein, G Listening to Prozac but hearing placebo: A meta analysis of antidepressant medication. Prevention & Treatment. 1, Article 0002a, No Pagination Specified 2. Kirsch, I Are drug and placebo effects in depression additive? Biological Psychiatry 47, Kirsch, I, Moore, TJ, Scoboria, A, Nicholls, SS The emperor's new drugs: An analysis of antidepressant medication data submitted to the U.S. Food and Drug Administration. Prevention & Treatment. 5(1), No Pagination Specified 4. Rosenzweig S Some implicit common factors in diverse methods of psychotherapy: At last the Dodo said, Everybody has won and all must have prizes. Am J Orthopsychiatry 6: Shapiro DA & Shapiro D Meta-analysis of comparative therapy outcome studies: A replication and refinement. Psychol Bull 92:
45 References (continued) 6. Robinson LA, Berman JS, Neimeyer RA Psychotherapy for treatment of depression: A comprehensive review of controlled outcome research. Psychol Bull 108: Wampold BE, Mondin GW, Moody M, et al A meta-analysis of outcome studies comparing bona fide psychotherapies: Empirically, All must have prizes. Psychol Bull 122: Ahn H, Wampold BE Where oh where are the specific ingredients? A meta-analysis of component studies in counseling and psychotherapy. J Counsel Psychol 48: Chambless DL, Ollendick TH Empirically supported psychological interventions: Controversies and evidence. Annual Rev Psychol 52: Luborsky, L., Rosenthal, R., Diguer, L., et al The dodo bird verdict is alive and well--mostly. J. Psychotherapy Integration Vol 12(1) 32-57
46 References (continued) 11. Wampold BE The great psychotherapy debate: Models, Methods, and Findings. Mahwah NJ: Lawrence Erlbaum Associates. Wampold BE, Mondin GW, Moody M, et al A meta-analysis of outcome studies comparing bona fide psychotherapies: Empirically, All must have prizes. Psychol Bull 122: McCall, WA How to experiment in education. New York: McmIllan. 13. Fisher, RA The design of experiments. Edinburgh: Oliver and Boyd. 14. Gehan, E. & Lemark, NA Statistics in medical research: Developments in clinical trials. New York: Plenum Press. 15. Martindale C The therapist-as-fixed-effect fallacy in psychotherapy research. J Consult Clin Psychol 46:
47 References (continued) 16. Luborsky L, Crits-Christoph P, McLellan T, et al Do therapists vary much in their success? Findings from four outcome studies. Am J Orthopsychiatry 56: Crits-Christoph P, Baranackie K, Kurcias JS, et al Metaanalysis of therapist effects in psychotherapy outcome studies. Psychother Res 1: Crits-Christoph P, Mintz J Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. J Consul Clin Psychol 59: Wampold BE Methodological problems in identifying efficacious psychotherapies. Psychother Res 7:21-43, 20. Elkin I A major dilemma in psychotherapy outcome research: Disentangling therapists from therapies. Clin Psychol Sci Prac 6:10-32.
48 References (continued) 21. Wampold BE, Serlin RC The consequences of ignoring a nested factor on measures of effect size in analysis of variance designs. Psychol Methods 4: Huppert JD, Bufka LF, Barlow DH, et al Therapists, therapist variables, and cognitive-behavioral therapy outcomes in a multicenter trial for panic disorder. J Consul Clin Psychol 69: Okiishi J, Lambert MJ, Nielsen SL, et al Waiting for supershrink: An empirical analysis of therapist effects. Clin Psychol Psychother 10: Brown GS, Jones ER, Lambert MJ, et al Identifying highly effective psychotherapists in a managed care environment. Am J Managed Care 11(8): Wampold BE, Brown GS Estimating variability in outcomes due to the therapist: A naturalistic study of outcomes in managed care. J Consul Clin Psychol. 73(5):
49 References (continued) 26. Elkin, I, Shae, T, Watkins, JT., et al National Institute of Mental Health Treatment of Depression Collaborative Research Program: General effectiveness of treatments. Archive of General Psychiatry. 46: Kim DM, Wampold BE, Bolt DM Therapist effects and treatment effects in psychotherapy: Analysis of the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Psychother Res. 16(2): McKay, KM, Imel, ZE & Wampold, BE. In press. Psychiatrist effects in the pharmacological treatment of depression. J. Affective Disorders. 29. Hannan C, Lambert MJ, Harmon C et al A lab test and algorithms for identifying clients at risk for treatment failure. J Clin Psychol 61(2): Lambert MJ, Harmon C, Slade K et al Providing feedback to psychotherapists on their patients progress: Clinical results and practice suggestions J Clin Psychol 61(2):
50 References (continued) 31. Harmon C, Hawkins, Lambert MJ et al Improving outcomes for poorly responding clients: The use of clinical support tools and feedback to clients. J Clin Psychol 61(2): Brown GS, Jones DR Implementation of a feedback system in a managed care environment: What are patients teaching us? J Clin Psychol 61(2): Claiborn CD, Goodyear EK Feedback in psychotherapy. J Clin Psychol 61(2): Lueger RJ Using feedback on patient progress to predict the outcome of psychotherapy. J Clin Psychol 54: Lambert MJ, Whipple JL, Smart DW, et al The effects of providing therapists with feedback on patient progress during psychotherapy: Are outcomes enhanced? Psychother Res 11(1):49-68.
51 References (continued) 36. Lambert MJ, Whipple JL, Vermeersch DA, et al Enhancing psychotherapy outcomes via providing feedback on client progress: A replication. Clin Psychol Psychother 9: Whipple JL, Lambert MJ, Vermeersch DA, et al Improving the effects of psychotherapy: The use of early identification of treatment failure and problem-solving strategies in routine practice. J Counsel Psychol 50(1): Lambert MJ, Whipple JL, Hawkins EJ, et al Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clin Psychol Sci Prac 10: Bachelor, A., & Horvath, A. (1999). The therapeutic relationship. In M.A. Hubble, B.L. Duncan, and S.D. Miller (eds.). The Heart and Soul of Change: What Works in Therapy. Washington, D.C.: APA Press, Blatt, S. J., Zuroff, D.C., Quinlan, D.M., & Pilkonis, P. (1996). Interpersonal factors in brief treatment of depression: Further analyses of the NIMH Treatment of Depression Collaborative Research Program. J Consul Clin Psychol. 64,
52 References (continued) 41. Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research, and Practice, 16, Burns, D., & Nolen-Hoeksema, S. (1992). Therapeutic empathy and recovery from depression in cognitive-behavioral therapy: A structural equation model. J Consul Clin Psychol. 60, Connors, GJ, DiClemente, CC., Carroll, KM, et al The therapeutic alliance and its relationship to alcoholism treatment participation and outcome. J Consul Clin Psychol, 65(4), Horvath, A. O., & Symonds, B. D. (1991). Relation between working alliance and outcome in psychotherapy: A meta-analysis. J Consul Clin Psychol. 38, Krupnick, J., Sotsky, SM, Simmens, S et al The role of the therapeutic alliance in psychotherapy and pharmacotherapy outcome: Findings in the National Institute of Mental Health Treatment of Depression Collaborative Research Project. J Consul Clin Psychol., 64,
53 About the presenter G.S. (Jeb) Brown is a licensed psychologist with a Ph.D. from Duke University. He served as the Executive Director of the Center for Family Development from 1982 to He then joined United Behavioral Systems (an United Health Care subsidiary) as the Executive Director for of Utah, a position he held for almost six years. In 1993 he accepted a position as the Corporate Clinical Director for Human Affairs International (HAI), at that time one of the largest managed behavioral healthcare companies in the country. In 1998 he left HAI to found the Center for Clinical Informatics, a consulting firm specializing in helping large organizations implement outcomes management systems. Client organizations include PacifiCare Behavioral Health/ United Behavioral Health, Department of Mental Health for the District of Columbia, Accountable Behavioral Health Care Alliance, Resources for Living and assorted treatment programs and centers throughout the world. Dr. Brown continues to work as a part time psychotherapist at behavioral health clinic in Salt Lake City, Utah. He does measure his outcomes.
54 Meadowmoor Rd. Salt Lake City, UT Voice
Optum ALERT Outcomes-Informed Treatment Bibliography
Optum ALERT Outcomes-Informed Treatment Bibliography Ahn, H., & Wampold, B.E. (2001). Where oh where are the specific ingredients? A metaanalysis of component studies in counseling and psychotherapy. Journal
More informationAchievements in Clinical Excellence (ACE) Measuring Effectiveness and Efficiency
Achievements in Clinical Excellence (ACE) Measuring Effectiveness and Efficiency November 2014 In 2009, Optum established the Campaign for Excellence (CFE) to measure clinical quality outcomes in the provider
More informationHere for Symposium on Therapeutic Relationships hope you are too.
Good morning. Here for Symposium on Therapeutic Relationships hope you are too. Lets get started. -I d like to tell you about plans for today -And introduce you to the team. 1 The Team: 1. Joshua: Therapeutic
More informationEvidence-Based Practice: Specific Methods and/or Therapeutic Relationship
Evidence-Based Practice: Specific Methods and/or Therapeutic Relationship Scott D. Miller, Ph.D. International Center for Clinical Excellence William R. Miller, Ph.D. International Center for Clinical
More informationPSYCHOTHERAPY : PROCESSES & OUTCOMES A
Syllabus PSYCHOTHERAPY : PROCESSES & OUTCOMES A - 51918 Last update 17-10-2014 HU Credits: 2 Degree/Cycle: 2nd degree (Master) Responsible Department: Psychology Academic year: 3 Semester: 1st Semester
More informationEstimating Variability in Outcomes Attributable to Therapists: A Naturalistic Study of Outcomes in Managed Care
Journal of Consulting and Clinical Psychology Copyright 005 by the American Psychological Association 005, Vol. 73, No. 5, 914 93 00-006X/05/$1.00 DOI: 10.1037/00-006X.73.5.914 Estimating Variability in
More informationUsing Client Feedback to Build a Strong Therapeutic Relationship
Using Client Feedback to Build a Strong Therapeutic Relationship Jeb Brown, Ph.D. Director, Center for Clinical Informatics Eric Hamilton, M.S. Vice President of Clinical Informatics, ValueOptions Overview
More informationCosette Taillac, LCSW National Strategic Leader for Mental Health & Wellness. 1 Copyright 2018 Kaiser Foundation Health Plan, Inc.
Kaiser Permanente Mental Health & Wellness Using Patient Reported Outcomes to Promote Shared Decision Making, Privilege the Patient Voice, and Deliver Exceptional Care Cosette Taillac, LCSW National Strategic
More informationIn Norcross, J., & Levant, R., & Beutler, L. (2005). Evidence-based practices in
Running Head: THE MANUAL IS NOT THE TERRITORY In Norcross, J., & Levant, R., & Beutler, L. (2005). Evidence-based practices in mental health: Debate and dialogue on the fundamental questions. Washington,
More informationOpening up to a total stranger and revealing some of your most intimate thoughts and feelings
CHOOSING A PSYCHOTHERAPIST AND BEING IN TREATMENT (Adapted from my manuscript, Failed Parental Love and the Lost Self) Opening up to a total stranger and revealing some of your most intimate thoughts and
More informationPsychotherapy: Shifting from Technique to Client. In reviewing recent research findings, the Task Force on Evidence-Based Therapy Relationships
Psychotherapy: Shifting from Technique to Client In reviewing recent research findings, the Task Force on Evidence-Based Therapy Relationships commissioned by the American Psychological Association, concluded,
More informationSPECIAL FEATURE. Paul A. Pilkonis University of Pittsburgh
Journal of Consulting and Clinical Psychology 1996. Vol. 64, No. 6, 1276-1284 Copyright 1996 by the American Psychological Association, Inc. 22-6X/96/$3. SPECIAL EATURE Characteristics of Effective Therapists:
More informationOutcomes of Client-Based Feedback: Comments and a third option
Digital Commons @ George Fox University Faculty Publications - Graduate School of Clinical Psychology Graduate School of Clinical Psychology 4-1-2011 Outcomes of Client-Based Feedback: Comments and a third
More informationAN INVESTIGATION OF SELF-ASSESSMENT BIAS IN MENTAL HEALTH PROVIDERS 1
Psychological Reports, 2012, 110, 2, 639-644 Psychological Reports 2012 AN INVESTIGATION OF SELF-ASSESSMENT BIAS IN MENTAL HEALTH PROVIDERS 1 STEVEN WALFISH Independent Practice, Atlanta, Georgia BRIAN
More informationNICE Guidelines in Depression. Making a Case for the Arts Therapies. Malcolm Learmonth, Insider Art.
1 NICE Guidelines in Depression. Making a Case for the Arts Therapies. Malcolm Learmonth, Insider Art. These notes are derived from a review of the full Guidelines carried out by Malcolm Learmonth, May
More informationLISTENING TO CLIENTS, NOT LABELS
2/16/2015 LISTENING TO CLIENTS, NOT LABELS Outcome-Informed Practice with Children and Families JACQUELINE SPARKS, PH.D. UNIVERSITY OF RHODE ISLAND jsparks@uri.edu LISTENING TO CLIENTS, NOT LABELS Outcome-Informed
More informationPsychotherapy Outcomes Research. Bob Bertolino, Ph.D.
Psychotherapy Outcomes Research Bob Bertolino, Ph.D. Primary Research Agendas Primary Agendas in Research and Practice 1. Empirically-Supported Treatments (ESTs); Evidence-Based Practices (EBPs) 2. Empirically-Supported
More informationLecture 4: Evidence-based Practice: Beyond Colorado
Lecture 4: Evidence-based Practice: Beyond Colorado A. Does Psychotherapy Work? Many of you are in the course because you want to enter the helping professions to offer some form of psychotherapy to heal
More informationAims of talk. Aims of talk. Overall effectiveness. Reviewing what we know. Does therapy work?
Reviewing what we know Counselling and Psychotherapy research findings: What we know and where we re going Aims of talk 1. Review what we know about the effectiveness of therapy, and the factors that make
More informationCognitive Therapy for Suicide Prevention
This program description was created for SAMHSA s National Registry for Evidence-based Programs and Practices (NREPP). Please note that SAMHSA has discontinued the NREPP program and these program descriptions
More informationThe Session Rating Scale: Preliminary Psychometric Properties of a Working Alliance Measure
JBT 3(1) 3-12 12/14/04 3:53 PM Page 3 Journal of Brief Therapy Volume 3, Number 1 Fall/Winter 2003 The Session Rating Scale: Preliminary Psychometric Properties of a Working Alliance Measure Barry L. Duncan,
More informationMEASUREMENT-BASED CARE
MEASUREMENT-BASED CARE Tracey Smith, Ph.D. Associate Director for Improving Clinical Care, South Central MIRECC President-Elect, AVAPL Lisa K. Kearney, Ph.D., ABPP Lead Mental Health Hiring Initiative,
More informationThe Yearning for Connection
The Yearning for Connection IACP Inaugural Conference: Strengthening Therapeutic Relationships Dr. Marcella Finnerty, IICP Friday October 16th, 2015 Carl Rogers Plant, Nurture, Grow and Harvest? Yearning
More informationThe Practitioner Scholar: Journal of Counseling and Professional Psychology Volume 7, 2018
144 Client Engagement Related to Their Satisfaction with Treatment Outcomes Huma A. Bashir, Wright State University Josephine F. Wilson, Wright State University and Greta H. Meyer, Mental Health and Recovery
More informationTherapeutic Relationship and Outcome Effectiveness: Implications for Counselor Educators
The Journal of Counselor Preparation and Supervision Volume 9 Number 2 Article 7 2017 Therapeutic Relationship and Outcome Effectiveness: Implications for Counselor Educators Nicole A. Stargell University
More informationCommentary on Beutler et al. s Common, Specific, and Treatment Fit Variables in Psychotherapy Outcome
Commentary on Beutler et al. s Common, Specific, and Treatment Fit Variables in Psychotherapy Outcome Kenneth N. Levy Pennsylvania State University In this commentary, the author highlights innovative
More informationThe Partners for Change Outcome Management System
The Partners for Change Outcome Management System Scott D. Miller and Barry L. Duncan Institute for the Study of Therapeutic Change Ryan Sorrell Resources for Living George S. Brown Center for Clinical
More informationThe meaningful use of routine outcome monitoring in a low intensity service for children and young people with anxiety disorders and depression
School of Psychology and Clinical Language Sciences The meaningful use of routine outcome monitoring in a low intensity service for children and young people with anxiety disorders and depression Polly
More informationAdult Theories and Techniques of Psychotherapy, 2009
University of Massachusetts Amherst ScholarWorks@UMass Amherst Graduate Training Psychological Services Center Spring 2009 Adult Theories and Techniques of Psychotherapy, 2009 Michael Constantino University
More informationOverview. An Old But Important Dilemma 10/24/12. The Role of the Clinician in Creating Confidence in Care
The Role of the Clinician in Creating Confidence in Care Theresa B. Moyers, Ph.D. Department of Psychology Center on Alcoholism, Substance Abuse and Addictions (CASAA) University of New Mexico Overview
More informationInstructions for the Short Assessment System
Instructions for the Short Assessment System The SAS is more than a diagnostic tool. It is a form of therapeutic assessment, which means that it serves as a useful intervention while also collecting information.
More informationThe Evolution of Psychotherapy: Training. Scott D. Miller, Ph.D. International Center for Clinical Excellence
Training Scott D. Miller, Ph.D. International Center for Clinical Excellence Training Time: M.S., MSW, 2-3 years; Psy.D. or Ph.D., 5-7 years. Money: Debt: Public, $10,500/year; Private $24,000/year Range
More informationDose-Response Studies in Psychotherapy
Dose-Response Studies in Psychotherapy Howard, K. I., Kopta, S. M., Krause, M. S., & Orlinsky, D. E. (1986). The dose-effect relationship in psychotherapy. The American Psychologist, 41, 159 164. Investigated
More informationNote: The trainings below represent a foundational list, and may be adapted based on audience and need.
MOTIVATIONAL INTERVIEWING Introduction to Motivational Interviewing (offered in English and Spanish) 2-day Course (12-14 credit hours) This course is designed to introduce clinicians and staff members
More informationNote: The trainings below represent a foundational list, and may be adapted based on audience and need.
MOTIVATIONAL INTERVIEWING Introduction to Motivational Interviewing (offered in English and Spanish) 2-day Course (12-14 credit hours) This course is designed to introduce clinicians and staff members
More informationContinuing Care. Part 3 Telephone Monitoring
Continuing Care Part 3 Telephone Monitoring The goal of telephone monitoring is to establish a consistent avenue for tracking a client s recovery process. Through telephone collaborations the addiction
More informationPsychosocial Services The Good
The Partners for Change Outcome Management System Evidence Based Practice One Client at a Time What s Up: PCOMS Research, Predictors, Implications 1. Research-proven strategy to address the problems of
More informationDo we Know When our Clients Get Worse? An Investigation of Therapists Ability to Detect Negative Client Change
Clinical Psychology and Psychotherapy Clin. Psychol. Psychother. 17, 25 32 (2010) Published online 13 November 2009 in Wiley InterScience (www.interscience.wiley.com)..656 Do we Know When our Clients Get
More informationAn Oxymoron. Scott D. Miller, Ph.D. International Center for Clinical Excellence
An Oxymoron Scott D. Miller, Ph.D. International Center for Clinical Excellence www.centerforclinicalexcellence.com #EVO2013 @theicce http://twitter.com/scott_dm http://www.linkedin.com/in/scottdmphd Ev
More informationProblem Gambling Foundation of New Zealand
Problem Gambling Foundation of New Zealand Outcome Measures When Results Matter Partners for Change Outcome Management System Laurie Siegel-Woodward and Ifo Fa anana Problem Gambling Foundation of New
More informationA Direction from COMPASS. Creating a context for change
A Direction from COMPASS Creating a context for change COMPASS and Outcomes Howard, et al. (1996) Managing as opposed to measuring The clinical success A commercial failure The Working Alliance as Performance
More informationSupportive psychotherapy and defense mechanisms: A Comment on the case of Matilde
43 Supportive psychotherapy and defense mechanisms: A Comment on the case of Matilde Jonathan Petraglia 1 The Case of Matilde provides us with an important example of how the in-depth study of a single
More informationPerformance Improvement Project Implementation & Submission Tool
PLANNING TEMPLATE Performance Improvement Project Implementation & Submission Tool INTRODUCTION & INSTRUCTION This tool provides a structure for development and submission of Performance Improvement Projects
More informationAny correspondence concerning this service should be sent to The Strathprints Administrator:
Cooper, Mick (2009) What do we know about the effectiveness of counselling and psychotherapy? In: Newport Centre for Counselling Research Conference (NCCR), 2009-11-14. (Unpublished), This version is available
More informationThe therapeutic alliance and psychotherapy outcomes for young adults aged 18 to
Title registration for a review proposal: The therapeutic alliance and psychotherapy outcomes for young adults aged 18 to 25 To start a Campbell review, a title must be registered and approved by the appropriate
More informationBenchmarking therapists: furthering the benchmarking method in its application to clinical practice
Qual Quant (2012) 46:1699 1708 DOI 10.1007/s11135-011-9548-4 Benchmarking therapists: furthering the benchmarking method in its application to clinical practice Takuya Minami G.S Brown Joyce McCulloch
More informationThe Treatment Effectiveness of Asynchronous Text Therapy for Depression and Anxiety: A Longitudinal Cohort Study
The Treatment Effectiveness of Asynchronous Text Therapy for Depression and Anxiety: A Longitudinal Cohort Study Thomas D. Hull Philippa Connolly Kush Mahan Katie Yang Abstract Background: Barriers to
More informationTailoring the Therapy Relationship to the Individual Client Evidence-Based Practices
Tailoring the Therapy Relationship to the Individual Client Evidence-Based Practices John C. Norcross, PhD norcross@scranton.edu Workshop Description Psychotherapy will maximize its effectiveness by targeting
More informationCognitive Behavioral Therapy For Late Life Depression A Therapist Manual
Cognitive Behavioral Therapy For Late Life Depression A Therapist Manual Anxiety likely contributes to poorer adaptive functioning, quality of life, and Using CBT to treat depression after stroke was first
More informationOutcome Research Coding Protocol. Coding Studies and Rating the Level of Evidence for the. Causal Effect of an Intervention
Outcome Research Coding Protocol Coding Studies and Rating the Level of Evidence for the Causal Effect of an Intervention National Panel for Evidence-Based School Counseling June, 2005 School counseling
More informationVPS PRACTICUM STUDENT COMPETENCIES: SUPERVISOR EVALUATION VPS PRACTICUM STUDENT CLINICAL COMPETENCIES
VPS PRACTICUM STUDENT COMPETENCIES: SUPERVISOR EVALUATION VPS PRACTICUM STUDENT CLINICAL COMPETENCIES EXPECTED CLINICAL COMPETENCIES OF UVM PRACTICUM STUDENTS COMPETENCY #1: TO DEVELOP EFFECTIVE COMMUNICATION
More informationPatient Predictors of Response to Interpersonal Psychotherapy (IPT) for Depression
Graduate Student Journal of Psychology Copyright 2006 by the Department of Counseling & Clinical Psychology 2006, Vol. 8 Teachers College, Columbia University ISSN 1088-4661 Patient Predictors of Response
More informationUsing Clinical Trials to Benchmark Effects Produced in Clinical Practice
Quality & Quantity (2008) 42:513 525 Springer 2006 DOI 10.1007/s11135-006-9057-z Using Clinical Trials to Benchmark Effects Produced in Clinical Practice TAKUYA MINAMI 1,, RONALD C. SERLIN 2, BRUCE E.
More informationThe Gentle Art of Saying No : How to Establish Appropriate Boundaries with Chronic Pain Patients
The Gentle Art of Saying No : How to Establish Appropriate Boundaries with Chronic Pain Patients DAVID COSIO, PHD, ABPP Biography David Cosio, PhD, is the psychologist in the Pain Clinic and the CARFaccredited,
More informationCopyright American Psychological Association. Introduction
Introduction Personality disorders (PDs) are commonly encountered in practice, but their management is challenging. Patients with these diagnoses can be described as the stepchildren of the mental health
More informationThe Personal and Professional Characteristics of Master Therapists and Matched Controls
University of Southern Maine USM Digital Commons All Theses & Dissertations Student Scholarship 2014 The Personal and Professional Characteristics of Master Therapists and Matched Controls Barbara A. Granville
More informationThe Wellness Assessment: Global Distress and Indicators of Clinical Severity May 2010
The Wellness Assessment: Global Distress and Indicators of Clinical Severity May 2010 Background Research has shown that the integration of outcomes measurement into clinical practice is associated with
More informationA new scale for the assessment of competences in Cognitive and Behavioural Therapy. Anthony D. Roth. University College London, UK
A new scale for the assessment of competences in Cognitive and Behavioural Therapy Anthony D. Roth University College London, UK Abstract Background: Scales for assessing competence in CBT make an important
More informationPractice-Based Research for the Psychotherapist: Research Instruments & Strategies Robert Elliott University of Strathclyde
Practice-Based Research for the Psychotherapist: Research Instruments & Strategies Robert Elliott University of Strathclyde Part 1: Outcome Monitoring: CORE Outcome Measure 1 A Psychological Ruler Example
More informationBringing hope and lasting recovery to individuals and families since 1993.
Bringing hope and lasting recovery to individuals and families since 1993. "What lies behind us and what lies before us are tiny matters compared to what lies within us." Ralph Waldo Emerson Our Statement
More informationIthink that randomized controlled trials (RCTs) are highly
Are Randomized Controlled Trials Relevant to Clinical Practice? Steven D Hollon, PhD 1 ; Bruce E Wampold, PhD, ABPP 2 Can J Psychiatry. 2009;54(9):637 643. Randomized Controlled Trials Are Relevant to
More informationUsing Measurement-Based Care to Enhance Substance Abuse and Mental Health Treatment
Using Measurement-Based Care to Enhance Substance Abuse and Mental Health Treatment TONMOY SHARMA MBBS, MSC CHIEF EXECUTIVE OFFICER SOVEREIGN HEALTH GROUP The Problem 5 to 10% of adults and 14 to 25% of
More informationThe Therapeutic relationship. Tom O Brien
The Therapeutic relationship Tom O Brien The role of the herbalist is to help facilitate people to heal themselves. While it is true that we are not psychotherapists the therapeutic relationship and communication
More informationPracticum Course/Group Supervision: Wednesdays 9-11 AM Individual Supervision: 1-hr per week to be scheduled b/w instructor and each student
Common Syllabus for Psyc6820/Psyc6830 Practicum/Advanced Practicum 1 st to 3 rd year practicum courses in Fall and Spring seemster Counseling Psychology PhD Program University of North Texas Practicum
More informationEvidence for The Therapeutic Relationship as the Primary Agent of Change. Bob Werstlein PhD Daymark Recovery Services
Evidence for The Therapeutic Relationship as the Primary Agent of Change Bob Werstlein PhD Daymark Recovery Services Factors that Account for Successful Outcomes Client/Extra Therapeutic- 40% Relationship-30%
More informationA New Therapy for Each Patient
A New Therapy for Each Patient Evidence-Based Methods of Personalizing Psychotherapy John C. Norcross, PhD, ABPP Workshop Description Psychotherapy will maximize its effectiveness by targeting the most
More informationJanuary 2, Dear Technical Review Committee Members:
Glenn A. Ally, Ph.D., M.P. (A Professional Psychology Corporation) Clinical Neuropsychologist Advanced Practice Medical Psychologist 155 Hospital Drive, Suite 200 Lafayette, Louisiana 70503 (337) 235-8304
More informationCopyright 1997 by the American Psychological Association, Inc. 1997, Vol. 122, No. 3, /97/$3.00
Psychological Bulletin Copyright 1997 by the American Psychological Association, Inc. 1997, Vol. 122, No. 3, 226-230 0033-2909/97/$3.00 The Flat Earth as a Metaphor for the Evidence for Uniform Efficacy
More informationPsychotherapy research: do we know what works for whom?
Fonagy, P; (2010) Psychotherapy research: do we know what works for whom? BRIT J PSYCHIAT, 197 (2) 83-85. 10.1192/bjp.bp.110.079657. EDITORIAL Psychotherapy research: do we know what works for whom? Peter
More informationCalculating clinically significant change: Applications of the Clinical Global Impressions (CGI) Scale to evaluate client outcomes in private practice
University of Wollongong Research Online Faculty of Health and Behavioural Sciences - Papers (Archive) Faculty of Science, Medicine and Health 2010 Calculating clinically significant change: Applications
More informationBlending Addiction Science and Practice: Evidence-Based Treatment and Prevention in Diverse Populations and Settings
William R. Miller, Ph.D. The University of New Mexico Have you noticed: that in multisite trials (like CTN) of treatments that are already evidence-based even under highly controlled, supervised, manual
More informationCollaborative Assessment and Management of Suicidality (CAMS)
This program description was created for SAMHSA s National Registry for Evidence-based Programs and Practices (NREPP). Please note that SAMHSA has discontinued the NREPP program and these program descriptions
More informationExploring the Contribution of Supervisors to Intervention Outcomes
Training and Education in Professional Psychology 2009 American Psychological Association 2009, Vol. 3, No. 2, 72 77 1931-3918/09/$12.00 DOI: 10.1037/a0014294 Exploring the Contribution of Supervisors
More informationThis webinar is presented by
Webinar DATE: Prostate Cancer effects on mental health November after 12, surgery 2008 Thursday, 1 st March 2018 Supported by The Royal Australian College of General Practitioners, the Australian Psychological
More informationDescription of intervention
Helping to Overcome PTSD through Empowerment (HOPE) Johnson, D., Zlotnick, C. and Perez, S. (2011) Johnson, D. M., Johnson, N. L., Perez, S. K., Palmieri, P. A., & Zlotnick, C. (2016) Description of Helping
More informationI T I S T I M E T O C H A N G E O U R P E R S P E C T I V E.
I T I S T I M E T O C H A N G E O U R P E R S P E C T I V E Goodbye to the JanteLaw a new focus in the work with children, young people, couples and families Psykolog Susanne Bargmann Teachingand implementingfit
More informationLearning from the Field s s Most Effective Practitioners Scott D. Miller, Ph.D. (Mild-mannered clinician and researcher)
Learning from the Field s s Most Effective Practitioners Scott D. Miller, Ph.D. (Mild-mannered clinician and researcher) 1 Supershrink: (n. soo-per-shrĭngk), slang 1. Unusually effective and talented psychotherapist;
More informationFocus of Today s Presentation. Partners in Healing Model. Partners in Healing: Background. Data Collection Tools. Research Design
Exploring the Impact of Delivering Mental Health Services in NYC After-School Programs Gerald Landsberg, DSW, MPA Stephanie-Smith Waterman, MSW, MS Ana Maria Pinter, M.A. Focus of Today s Presentation
More informationDistance Learning in Depression for Rural Primary Care and Mental Health Providers
Distance Learning in Depression for Rural Primary Care and Mental Health Providers Scott J. Adams, Psy.D. Mimi McFaul, Psy.D. Fran Dong, M.S. John Fortney, Ph.D. June 2009 Working Paper Western Interstate
More informationMembers recognize that other members share similar feelings,
Practice Guidelines for Group Psychotherapy http://www.agpa.org/guidelines/factorsandmechanisms.html (accessed 20 July 2012) Therapeutic Factors and Therapeutic Mechanisms Understanding mechanisms of action
More informationProgramme Name: Climate Schools: Alcohol and drug education courses
STUDY REFERENCE: C/ADEPIS01 Programme Name: Climate Schools: Alcohol and drug education courses Contact Details: Nicola Newton, University of New South Wales, email: n.newton@unsw.edu.au Natasha Nair,
More informationThe treatment of personality disorder: Where are we? Where do we go from here? Where do we want to end up?
The treatment of personality disorder: Where are we? Where do we go from here? Where do we want to end up? The rationale for integrated treatment JOHN LIVESLEY WHERE ARE WE? 1. What works? 2. What changes?
More informationPsychotherapy as mutual encounter: a study of therapeutic conditions
Loughborough University Institutional Repository Psychotherapy as mutual encounter: a study of therapeutic conditions This item was submitted to Loughborough University's Institutional Repository by the/an
More informationImproving clinicians' attitudes toward providing feedback on routine outcome assessments
University of Wollongong Research Online Faculty of Health and Behavioural Sciences - Papers (Archive) Faculty of Science, Medicine and Health 2009 Improving clinicians' attitudes toward providing feedback
More informationFREE IN-DEPTH TRAINING
FREE IN-DEPTH TRAINING Become a facilitator in After Deployment: Adaptive Parenting Tools (ADAPT) WHAT: Train to become a facilitator in ADAPT, a treatment and prevention intervention for military families
More informationMaximizing Treatment Effectiveness in Clinical Practice: An Outcome-Informed, Collaborative Approach
CLINICAL EFFECTIVENESS Maximizing Treatment Effectiveness in Clinical Practice: An Outcome-Informed, Collaborative Approach Michael L. Saggese ABSTRACT Clinicians need easy-to-use, practical, systematic
More informationUNIVERSITY OF WASHINGTON PSYCHIATRY RESIDENCY PROGRAM COGNITIVE-BEHAVIORAL THERAPY (CBT) COMPETENCIES
UNIVERSITY OF WASHINGTON PSYCHIATRY RESIDENCY PROGRAM COGNITIVE-BEHAVIORAL THERAPY (CBT) COMPETENCIES Knowledge The resident will demonstrate: The ability to articulate the key principles related to cognitive-behavioral
More informationHeidi Clayards Lynne Cox Marine McDonnell
Heidi Clayards Lynne Cox Marine McDonnell Introduction to Interpersonal Psychotherapy (IPT) Adaptations from IPT to IPT-A Theoretical framework Description of treatment Review of the manual and demonstration
More informationIMPACT OF COMMON FACTORS ON PSYCHOTHERAPY OUTCOMES
IMPACT OF COMMON FACTORS ON PSYCHOTHERAPY OUTCOMES University of East Anglia Psychology with Sociology BSc 1316 City College Norwich ACKNOWLEDGEMENTS Thank you for my supervisor, whose generous guidance
More informationPsychometric Evaluation of the Brief Adjustment Scale-6 (BASE-6): A New Measure of General. Psychological Adjustment. Alexandra Paige Peterson
Psychometric Evaluation of the Brief Adjustment Scale-6 (BASE-6): A New Measure of General Psychological Adjustment Alexandra Paige Peterson A thesis submitted in partial fulfillment of the requirements
More information1.01. Helping is a broad and generic term that includes assistance provided by a variety of
CHAPTER 1: INTRODUCTION TO HELPING Multiple-Choice Questions 1.01. Helping is a broad and generic term that includes assistance provided by a variety of individual such as: a. friends b. family c. counselors
More informationMaximizing Psychotherapy Outcome beyond Evidence-Based Medicine
Special Article Psychother Psychosom 2017;86:80 89 Received: December 7, 2016 Accepted after revision: December 17, 2016 Published online: February 10, 2017 Maximizing Psychotherapy Outcome beyond Evidence-Based
More informationTrigger. Myths About the Use of Medication in Recovery BUPRENORPHINE TREATMENT: A TRAINING FOR MULTIDISCIPLINARY ADDICTION PROFESSIONALS
BUPRENORPHINE TREATMENT: A TRAINING FOR MULTIDISCIPLINARY ADDICTION PROFESSIONALS Module VI Counseling Buprenorphine Patients Myths About the Use of Medication in Recovery! Patients are still addicted!
More informationFEEDBACK INFORMED TREATMENT
CONTACT David S. Prescott, LICSW FEEDBACK INFORMED TREATMENT Clinical Director and Director of Professional Development and Quality Improvement Becket Family of Services 2018 David S. Prescott, LICSW Welcome!
More informationDiscussion of meta-analysis on the benefits of psychotherapy
Discussion of meta-analysis on the benefits of psychotherapy Meta-analyses of psychodynamic long term psychotherapy. Science or lobbyism? Peter Wilhelm 27.4.2016 Overview of Today s Lecture Critical Discussion
More informationTELEPSYCHOLOGY FOR THE PSYCHOLOGIST IN PRIVATE PRACTICE
TELEPSYCHOLOGY FOR THE PSYCHOLOGIST IN PRIVATE PRACTICE Dr. Madalina Sucala Icahn School of Medicine at Mount Sinai Department of Oncological Sciences OUTLINE Video conferencing as a delivery method Assessment
More informationDocumenting the Effectiveness of Psychotherapeutic Interventions
VoLuMe 1 number 1 MaRCh 2010 Documenting the Effectiveness of Psychotherapeutic Interventions It Is Up to Psychologists to Interpret and Give True Meaning to the Empirical Evidence Dr. Martin Drapeau,
More informationResponsiveness in psychotherapy research and practice. William B. Stiles Jyväskylä, Finland, February 2018
Responsiveness in psychotherapy research and practice William B. Stiles Jyväskylä, Finland, February 2018 Responsiveness Behavior influenced by emerging context (i.e. by new events),
More informationAlliance and Outcomes in Counseling Canadian Aboriginal Peoples
University of Montana ScholarWorks at University of Montana Theses, Dissertations, Professional Papers Graduate School 2013 Alliance and Outcomes in Counseling Canadian Aboriginal Peoples Sidney Shaw The
More information