Evidence-Based Treatment: Where Is the Evidence? A review of the video Evidence-Based Treatment with Larry Beutler Washington, DC: American Psychological Association, 2007. American Psychological Association Systems of Psychotherapy Video Series I, Item no. 4310794. $99.95 Reviewed by Thomas H. Ollendick Matthew A. Jarrett The movement to develop, identify, and disseminate evidence-based treatments has been contentious. On the surface, it hardly seemed possible that anyone could or would object to the initial report issued by the Society of Clinical Psychology (Division 12) of the American Psychological Association (APA) in 1995 or anticipate that the movement associated with it would become so controversial. Surely, identifying, developing, and disseminating treatments that have empirical support should be encouraged, not discouraged, especially by a profession that is committed to the welfare of those whom it serves. Sensible as this may seem, the task force report was not only controversial; moreover, and unfortunately, it served to deeply divide the profession of clinical psychology and related mental health disciplines (Chambless & Hollon, 1998; Chambless & Ollendick, 2001).
It seems to us that the task force report was long overdue. In the last 25 30 years, important developments including advances in our understanding of diverse psychopathologies, innovative assessment and treatment practices, and exciting experimental and longitudinal designs for the study of treatment process and outcome have occurred in the treatment arena. These advances have resulted in hundreds of studies in both child and adult treatment research, captured in volumes edited by Nathan and Gorman (2002) and Barrett and Ollendick (2004). These volumes, titled A Guide to Treatments That Work and Handbook of Interventions That Work With Children and Adolescents, respectively, attest to the voluminous literature and to the advances that have been made. At this time, there is little doubt that psychotherapy can result in beneficial impacts on the lives of adults as well as children and their families. These volumes clearly demonstrate that psychotherapy outperforms wait-list and attention-placebo control conditions; moreover, in several studies, but not all, it is evident that some forms of psychotherapy work better for some disorders than do other psychotherapies. As a result, much progress has been made, and we can confidently assert that the field of psychotherapy research has moved beyond the simple question of asking whether psychotherapy works to identifying the efficacy of specific treatments for individuals who present with specific behavioral, emotional, and social problems. This is an exciting time in the field of psychotherapy research. Although recent reviews of evidence-based treatments reveal that several efficacious treatments have been identified, we must insert a word of caution. It is evident that many, but not all, of the treatments shown to be efficacious for a host of disorders have been behavioral and cognitivebehavioral ones. Unfortunately, we do not really know whether other frequently practiced treatments from other theoretical orientations work or do not work; for the most part, they simply have not been evaluated sufficiently. Still, the value of developing, identifying, and promulgating treatments that do have support for their use seems intuitively desirable and necessary. The demonstration of the efficacy of treatments in controlled settings may point the way to determining the effectiveness of the treatments in real-life clinical settings. These developments notwithstanding, many of those in the practice and treatment research communities have expressed the belief that the movement to identify evidence-based treatments ignores important aspects of relationshipbased therapies, in particular the role of the therapeutic or working alliance and the role of patient and therapist factors in predicting treatment outcomes. To
address this concern, a task force was created by the Division of Psychotherapy (Division 29) of APA, resulting in the now-seminal task force report edited by John Norcross (2002) and titled Psychotherapy Relationships That Work. This task force report focused on relationship and patient therapist factors and ignored potentially important treatment-specific findings. Thus, in our opinion, it erred in a different direction from the Division 12 Task Force: Namely, by not examining how relationship and patient therapist factors intersect with specific treatments and the theoretical models that underlie these treatments, it, too, inevitably possessed significant shortcomings. Surely, although relationship factors might be important for all psychotherapies, they might be more or less important for some forms of psychotherapy than others on the basis of differences in underlying psychopathologies and problems being treated. These two task force reports have come to be viewed as somewhat conflicting and contradictory of one another. In response to this state of affairs, Castonguay and Beutler formed yet a third task force: the Task Force on Empirically Based Principles of Psychotherapeutic Change. This task force was jointly sponsored by the North American Society for Psychotherapy Research and Division 12 (Society of Clinical Psychology) of the APA. It attempted to synthesize and expand upon the two existing task force reports by identifying a set of evidence-based principles that would guide psychotherapy research and practice. For this purpose, a principle was defined by Castonguay and Beutler as the conditions under which a concept (participant, relationship quality, or intervention) will be effective and an empirically based principle was described as one that reflects the role of participant characteristics, relationship qualities, or components of treatment that are found in the treatments identified by the Division 12 or Division 29 Task Force Reports, or that is supported by the preponderance of the available evidence (Castonguay & Beutler, 2006, p. 6). In the edited volume that resulted from this task force report, contributors of diverse theoretical orientations produced individual chapters that examined participant, relationship, and technique factors for four commonly occurring psychiatric disorders: dysphoria, anxiety, personality disorders, and substance use disorders. Each section was concluded with a chapter that integrated the three therapeutic factors and put forth common and unique principles that could guide the evidence-based treatment of that disorder. For example, the integrative chapter for dysphoric disorders included common principles of therapeutic change applied to participant factors (e.g., outcome is not substantially or meaningfully enhanced by variations in patient gender), common principles related to the therapeutic relationships (e.g., therapist
positive regard is a probable contributor to patient benefit), common principles of selecting techniques and interventions (e.g., facilitating client selfexploration can be useful), and unique principles for treating depression and dysphoria (e.g., age is a negative predictor of a patient's response to general psychotherapy). Overall, this approach can be viewed as a pragmatic one that emphasizes common principles that are associated with positive treatment outcome rather than an approach that is tied to a specific theoretical orientation (e.g., psychodynamic or cognitive-behavioral). The DVD under review, Evidence-Based Treatment (with Larry Beutler), is based on the Castonguay and Beutler volume and is intended to illustrate evidence-based treatment at work. In the first section of the DVD, part of the APA Systems of Psychotherapy Series, Jon Carlson interviews Beutler about the nature of evidence-based treatment. Beutler briefly sketches out the work of the task force and elaborates upon some of the principles of this approach. For this purpose, a pamphlet or guide would have been extremely helpful, particularly for delineating the definition of empirically supported principles and how they were derived. The discourse, though informative, is much too brief, and the rich background to this approach is shortchanged. In fact, we would suggest that the DVD ought to be accompanied by the 400-page volume edited by Castonguay and Beutler and published by Oxford University Press. In the absence of such background materials, an appreciation for the nuances and complexity of this approach is lost, and the DVD is somewhat like the proverbial emperor without clothes. It is difficult for any DVD to stand alone. In the second part of the DVD, Beutler demonstrates his approach to evidence-based treatment with a 20-year-old male who presents with problems of depression, drug abuse, and persistent self-esteem issues. Prior to the brief therapy session, Dr. Beutler introduces the process of evidence-based assessment, as he provides the host with copies of the output from the initial assessment. It is difficult to see the details of this output on the DVD, so it would have been helpful for these materials to accompany the DVD. In particular, seeing the assessment results for the current case would have been valuable in better understanding how this case was conceptualized prior to treatment and how that conceptualization led to the evidence-based treatments that were enlisted. After the short assessment review, Beutler conducts a brief therapy session (approximately 45 minutes) with the young man featured in the video. One of the useful technical aspects of the DVD is the on-screen clock that shows the time elapsed in the therapy session. In addition, the DVD also offers the option of narration by Beutler during the therapy session, a function that allows for a real-time assessment of the process of
treatment. In this clinically rich and sensitive demonstration, Beutler demonstrates his vast clinical acumen. As a therapist, he exudes relationship skills, and he embodies the working alliance at its best. He also shows that he is a wise and seasoned therapist as he engages in empathic listening, cognitive restructuring, processing of emotion, role-playing, and, yes, even some psychodynamic interpretations. The patient seems to benefit from this rich interchange. Following this brief therapy session, Beutler and Carlson discuss the session and its relevance to evidence-based treatment. The discerning reader might ask about the state of evidence for the various strategies that are employed. On the DVD, Beutler simply asserts that all of the things he does in treatment are evidence based and fails to link his approach to the evidence-based principles outlined in the task force report. One has to turn to the edited volume by Castonguay and Beutler (2006) to appreciate the evidence. In that volume, we can read that evidence does exist for using behavioral, cognitive-behavioral, interpersonal, brief psychodynamic, and process experiential psychotherapies in the treatment of dysphoric disorders (p. 85). We can also read that effective treatment is enhanced when therapists strive to develop and maintain a positive working alliance with their clients and that effective treatment is facilitated when therapist and patient share common goals of treatment and are collaborative in seeking to achieve these goals (p. 113). These are among nine evidence-based principles related to the therapeutic relationship that are highlighted. Yet, the discerning reader might also question the evidence for this principle-based approach from at least two different vantage points. First, the evidence presented in the Castonguay and Beutler (2006) volume on dysphoric disorders is largely applicable to major depressive disorder. In the DVD, upon query from Carlson, Beutler indicates that his working diagnosis is that of dysthymia. Given the long-standing duration of the dysphoria and the presence of low energy, low self-esteem, difficulty making decisions, and the patient's reported feelings of hopelessness, we would not disagree with this diagnosis. Yet, where is the evidence that the above-indicated participant, relationship, and technique factors work with dysthymia? Assuming that major depressive disorder and dysthymia are different disorders and their treatment might require a somewhat different approach, the evidence is not all that clear. It would be more comforting to see the evidence that illustrates that these two disorders can be effectively treated with the same evidence-based approach. Until such evidence is forthcoming, we would submit that the approach illustrated is not evidence based (using the criteria put forth by Castonguay and
Beutler). Second, although there is considerable evidence that behavioral, cognitive-behavioral, and interpersonal psychotherapies (and, to some extent, brief psychodynamic and process experiential psychotherapies) work with depressed patients, there is little or no evidence that an integrated amalgam of these interventions actually works with these patients. That is, as far as we are aware, there is no evidence available that what is labeled evidence-based treatment in this DVD actually works with these kinds of patients, or any kind of patients, for that matter. We would need to submit this principle-based approach to the same exacting criteria of the Division 12 and 29 task forces to determine if sufficient evidence exists. Therein lies the irony to this approach. In the absence of evidence, it cannot be viewed as evidence based. The evidence for the various principles espoused in this DVD and in the edited volume is embedded in approaches that are, for the most part, theory driven and theory contextualized. For example, the principle involving challenging specific dysfunctional thoughts and core negative beliefs is embedded in a cognitive-behavioral approach, whereas the principle indicating that a secure attachment pattern in the therapist appears to facilitate the treatment process is embedded in attachment theory. The utility of these principles combined in this decontextualized approach is not established. Again, where is the evidence? Principles out of context may not be the same as principles in context. Of course, our assertion needs to be systematically examined so too, however, must the assertion that they will be equally effective when decontextualized. As Castonguay and Beutler (2006) noted about principles in their text, It is indeed important to state that very few of them, if any, have been measured directly or found to be causally related to client's improvement in definite, experimental studies. They also note that rather than referring to them as being `empirically supported,' it might be more appropriate to define these principles as `empirically derived' or `empirically grounded' (p. 8). This semantic difference seems less important to us than is the need for additional evidence for this approach prior to labeling it as evidence-based treatment. Our concern with this approach (even though the senior author was a member of this task force and coauthor of the chapter on technique factors for the anxiety disorders) is twofold. First, from a scientific standpoint, it is unclear why an evidence-based approach has to abandon theory and be solely based on decontextualized empirical support. Why can we not take the results of the Division 12 and Division 29 task forces and synthesize them conceptually and theoretically? Think, for a moment, about how behavioral and cognitive-
behavioral treatments might be enriched by inclusion of important participant and relationship factors. Think, for another moment, about how more relationship-based psychotherapies might be embellished by technique factors from other theoretical persuasions that possess considerable evidentiary support (such as the behavioral and cognitive-behavioral ones). For example, motivational interviewing has proven to be an important adjunctive tool in the treatment of substance abuse. At the same time, this approach is not used solely in treatment, as it only addresses client motivation rather than the actual process of treatment once motivation is established. The principles outlined by Castonguay and Beutler (2006) could be similarly mapped onto treatment or relationship factors identified in the Division 12 and Division 29 task forces. Surely such integrated approaches have a place in our armamentarium; however, admittedly, they too will require careful systematic evaluation. Our second concern with this approach is a more applied and practical one. Granted, Beutler is an outstanding clinician; however, he is a clinician who has nearly 40 years of experience in terms of clinical research and practice. How would we go about teaching our trainees and less-experienced psychotherapists how to use alas, not just use, but master the various principles involved in conducting evidence-based treatment as it is presented in this DVD? Would they need to be conversant in behavioral, cognitivebehavioral, interpersonal, process experiential, and brief dynamic psychotherapies to treat a dysphoric 20-year-old? And would they then need to be wise and prescient enough to seek out and implement the various principles in a therapeutically savvy way and weave them into a rich tapestry? We suggest that this is most unlikely to be accomplished, let alone done in a manner that will result in efficacious outcomes. In conclusion, let us be clear: We are ardent supporters of evidencebased treatment. However, we believe that evidence-based treatment must be embedded into a theoretical context that provides the clinician with a framework a framework that guides her or him in a coherent, strategic, and thoughtful delivery of service that is intimately based on that theory of psychopathology, assessment, and treatment. We do not believe that the theoretical baby needs to be thrown out with the evidence-based bath water. Rather, we submit that a theoretically informed approach that is guided by the best available evidence about participant, relationship, and technique factors will serve our clients well. Such a treatment might also more likely be mastered by our trainees and less-experienced psychotherapists. However, we will need to let the evidence speak in this regard. Our proclamations alone will not suffice.
References Barrett, P. M., & Ollendick, T. H. (Eds.). (2004). Handbook of interventions that work with children and adolescents: Prevention and treatment. West Sussex, England: Wiley. Castonguay, L. G., & Beutler, L. E. (Eds.). (2006). Principles of therapeutic change that work. New York: Oxford University Press. Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66, 7 18. Chambless, D. L., & Ollendick, T. H. (2001). Empirically supported psychological interventions: Controversies and evidence. Annual Review of Psychology, 52, 685 716. Nathan, P. E., & Gorman, J. M. (2002). A guide to treatments that work (2nd ed.). New York: Oxford University Press. Norcross, J. C. (Ed.). (2002). Psychotherapy relationships that work: Therapist contributions and responsiveness to patient needs. New York: Oxford University Press. PsycCRITIQUES 1554-0138 August 29, 2007, Vol. 52, Release 35, Article 17 2007, American Psychological Association