Acupuncture for Depression?: A Systematic Review of Systematic Reviews Evaluation & the Health Professions 34(4) 403-412 ª The Author(s) 2011 Reprints and permission: sagepub.com/journalspermissions.nav DOI: 10.1177/0163278710386109 http://ehp.sagepub.com E. Ernst 1, Myeong Soo Lee 1,2, and Tae-Young Choi 2 Abstract Acupuncture is often advocated as a treatment for depression, and several trials have tested its effectiveness. Their results are contradictory and even systematic reviews of these data do not arrive at uniform conclusions. The aim of this review is to critically evaluate all systematic reviews of the subject with a view of assisting clinical decisions. Thirteen electronic databases were searched to identify all relevant articles. Data of these systematic reviews and the primary studies they included were extracted independently by the two authors according to predefined criteria. Eight systematic reviews including seventy-one primary studies were found. Five of the reviews arrived at positive conclusions and three did not. All the positive reviews and most of the positive primary studies originated from China. There are reasons to believe that these reviews are less than reliable. In conclusion, the effectiveness of acupuncture as a treatment of depression remains unproven and the authors findings are consistent with acupuncture effects in depression being indistinguishable from placebo effects. 1 Complementary Medicine, Peninsula Medical School, University of Exeter, Exeter, UK 2 Division of Standard Research, Korea Institute of Oriental Medicine, Daejeon, South Korea Corresponding Author: E. Ernst, Complementary Medicine, Peninsula Medical School, University of Exeter, 25 Victoria Park Road, Exeter EX2 4NT, UK Email: Edzard.Ernst@pms.ac.uk
404 Evaluation & the Health Professions 34(4) Keywords depression, acupuncture, alternative medicine, bias, evidence Acupuncture has been proposed as a nondrug therapy for depression, but skeptics are keen to point out that acupuncture lacks biological plausibility. Several trials, some of reasonably good quality, have tested the hypothesis that acupuncture alleviates the symptoms of depression (Ernst, Rand, & Stevinson, 1998). However, these data are highly contradictory and therefore they have the potential to confuse clinicians. In this situation, a systematic review might create clarity. In recent years, several systematic reviews of acupuncture for depression have emerged. Confusingly, they also arrive at dramatically different conclusions. Consequently, the existing systematic reviews are not a useful tool for assisting clinical decisions either. This article is aimed at summarizing the existing systematic reviews and at clarifying why their conclusions are contradictory. Our objective is to provide clinicians with the most reliable evidence about the therapeutic value of acupuncture in the management of depression. Method Electronic literature searches were conducted in Medline, Embase, Amed, CINAHL, Health Technology Assessments, DARE, the Cochrane Library, five Korean medical database (Korean Studies Information, DBPIA, Korea Institute of Science and Technology Information, KoreaMed, and Research Information Service System), and one Chinese database (CNKI) without restrictions of time or language. In addition, our departmental files were hand searched. The search terms were acupuncture, depression, systematic review, and meta-analysis. Abstracts of reviews thus located were inspected by two authors and those appearing meet the inclusion criteria were retrieved and read in full by two authors. Reviews were defined as systematic if they included an explicit and repeatable method for searching the scientific literature and if there were explicit and repeatable inclusion and exclusion criteria for studies. To be included, systematic reviews had to be concerned specifically with the effectiveness of acupuncture for depression and to include evidence from at least two controlled clinical trials. Systematic reviews were considered, regardless of the acupuncture types including body, ear or electro acupuncture, and so on. Systematic reviews of complex packages of intervention that happened to include acupuncture as one of several
Ernst et al. 405 elements were excluded. Reviews that depended on previous systematic reviews for their primary data were also excluded. All primary studies included in these systematic reviews were identified and key data were extracted from these trials. Data were extracted independently by two authors (EE and MSL) using predefined criteria (Tables 1 and 2). Disagreements were resolved by discussion between the authors. Judgments about the quality of the primary studies were taken from the respective systematic reviews. The Overview Quality Assessment Questionnaire (OQAQ) was used to evaluate the methodological quality of all included systematic reviews (Oxman & Guyat, 1991). The score ranges from 1 to 7; a score of 3 or less was considered as indicative of extensive or major flaws and a score of 5 or more as suggesting minor or minimal flaws. The two authors assessed the OQAQ independently and discrepancies were settled by discussion. Results Eight systematic reviews were found (Leo & Ligot, 2007; Mukaino, Park, White, & Ernst, 2005; Smith, Hay, & MacPherson, 2010; Sun, Chen, Gao, Xiong, 2008; Wang, Sun, Zou, & Zhang, 2008; Wang, Qi, et al., 2008; Xiong et al., 2009; Zhang, Chen, Yip, Ng, & Wong, 2010; Zhong, Huang, & Li, 2008). Key data are summarized in Table 1. Six reviews (Sun et al., 2008; Wang, Sun, et al., 2008; Wang, Qi, et al., 2008; Xiong et al., 2009; Zhang et al., 2010; Zhong et al., 2008) originated from China, one from the United States (Leo & Ligot, 2007), and two were multinational efforts (Mukaino et al., 2005; Smith et al., 2010). All of the Chinese reviews (Sun et al., 2008; Wang, Sun, et al., 2008; Wang, Qi, et al., 2008; Xiong et al., 2009; Zhang et al., 2010; Zhong et al., 2008) arrived at positive conclusions, while all of the non-chinese reviews failed to do so (Leo & Ligot, 2007; Mukaino et al., 2005; Smith et al., 2010). The reviews included between 7 and 30 randomized controlled trials (RCTs), and there was considerable overlap regarding the primary studies. In total, the reviews included 71 different primary studies (Table 2). Four reviews (Sun et al., 2008; Wang, Sun, et al., 2008; Xiong et al., 2009; Zhang et al., 2010) included exclusively or mostly Chinese studies, and all of these reviews reached positive conclusions about the effectiveness of acupuncture. Three of the Chinese reviews failed to include any of the RCTs that originated from outside China (Sun et al., 2008; Wang, Sun, et al., 2008; Xiong et al., 2009). The six positive (Sun et al., 2008; Wang, Sun, et al., 2008; Wang, Qi, et al., 2008; Xiong et al., 2009; Zhang et al.,
Table 1. Key Data From Systematic Reviews First Author (Year) Country Number of Primary RCTs (Country of Origin of ) Total Sample Size Study-High/moderate/poor quality) a Results of Study Positive/ Neutral/ Negative) Overall Conclusion (Quote) Meta-Analysis Smith (2010) Australia and UK Zhang (2009) China Xiong (2009) China 30 (China: 24; USA: 2; Germany: 2; Australia: 1; UK: 1) 35 (China: 33; USA:1; Germany: 1) 2,812 4/9/17 21/7/2... insufficient evidence to recommend the use of acupuncture for people with depression 3,678 6/23/6 17/15/3 Acupuncture therapy is safe and effective in treating major depressive disorder 9 (China:9) 903 0/2/7 5/2/2 Acupuncture is not inferior to western medicine and it is worth noting that acupuncture is associated with few adverse reactions Several meta-analyses were performed for different types of controls. The results show no significant differences between acupuncture and drug treatments or sham acupuncture. RR ¼ 1.06 (95% CI ¼ [0.97, 1.17]) compared with antidepressants Several meta-analyses were performed for different types of controls. The results show no significant differences between acupuncture and conventional drug therapy. Review b (OQAQ) Comment 7 Cochrane review. Superiority of acupuncture compared to waiting list could be due to nonspecific effects of acupuncture. 7 Discrepancy between meta-analytic result and conclusion 7 The conclusion regarding non-inferiority was not based on adequately designed non-inferiority studies. (continued) 406
Table 1 (continued) First Author (Year) Country Number of Primary RCTs (Country of Origin of ) Total Sample Size Study-High/moderate/poor quality) a Results of Study Positive/ Neutral/ Negative) Overall Conclusion (Quote) Meta-Analysis Wang (2008a) China Zhong (2008) China Wang (2008b) China 8 (China: 2; USA:3; Germany: 2; Australia:1) 7 (China: 4; USA: 2; Australia: 1) 477 4/3/1 7/0/1... acupuncture was an effective treatment that could significantly reduce the severity of disease in patients with depression 895 5/1/1 4/0/3 Based on current evidence, acupuncture is a promising treatment for depression 14 (China: 14) 1,317 1/3/10 5/5/4 Both acupuncture and medication possibly are effective for depression with good safety SMD ¼ 0.65 (95% CI ¼ [1.18, 0.11]) compared with sham acupuncture WMD ¼ 0.87 (95% CI ¼ [ 2.08, 0.35]) compared with western drugs WMD ¼ 0.93 (95% CI ¼ [ 1.48, 0.38]) compared with western drug OR ¼ 0.73 (95% CI ¼ [0.38, 1.40]) compared with western drugs Review b (OQAQ) Comment 6 Positive effect for improvement of depression but not for response rate 6 Cochrane review 6 Discrepancy between meta-analytic result and HAMD scores (continued) 407
Table 1 (continued) First Author (Year) Country Number of Primary RCTs (Country of Origin of ) Total Sample Size Study-High/moderate/poor quality) a Results of Study Positive/ Neutral/ Negative) Overall Conclusion (Quote) Meta-Analysis Review b (OQAQ) Comment Sun (2008) China 8 (China:8) 619 1/3/4 3/4/1 Acupuncture is not inferior to western medicine and it is worth noting that acupuncture is associated with few adverse reactions Leo (2007) US 9 (China: 5; USA: 2; Germany: 2) Mukaino (2005) Japan, US; UK 7 (China: 4; USA: 1; Germany: 2) 9 74 1/2/6 5/4/0... the evidence thus far is inconclusive 509 1/1/5 4/3/0 The evidence from controlled trials is insufficient... Several meta-analyses were performed for each drug. The results show no significant differences between acupuncture and conventional antidepressants 6 Only Chinese studies were included No 5 Jadad scores of some primary studies differed from those of Wang No 6 The first author s own review Note. RCT ¼ randomized controlled trials; SMD ¼ Standardized Mean Difference; WMD ¼ Weight Mean Difference. a primary studies was scored by Jadad score. Jadad score 4: having high quality; Jadad score ¼ 3: having moderate quality; Jadad score 2: having low quality. b OQAQ ¼ Overview Quality Assessment Questionnaire. The overall score is from 1 to 7. OQAQ 3: having extensive or major flaws; 5 OQAQ: having minor or minimal flaws. 408
Ernst et al. 409 2010; Zhong et al., 2008) and one of the negative reviews (Smith et al., 2010) used a meta-analytical approach. Most of the reviews seemed to be of reasonably good technical quality. There was considerable overlap between the systematic reviews in relation to the primary data included. Seventy one primary studies were included across the reviews. They tested acupuncture versus sham acupuncture (n ¼ 12), acupuncture versus drugs (n ¼ 54), and acupuncture versus waiting list or no treatment (n ¼ 5). Thirty-seven of these studies reported positive effects of acupuncture and twenty-five reported effects equivalent to those of drug treatments, while nine trials were negative. The quality of primary studies was high in 9 trials, moderate in 34 studies, and low in 28 trials. Most studies of them were conducted in China, three in the United States, two in Germany, and one each in Australia and the United Kingdom. Of the 64 primary studies originating from China, only 6 reported a negative result. Discussion Our analysis shows that a surprisingly high number of systematic reviews of acupuncture for depression have been published. Their conclusions about the therapeutic value of acupuncture for depression differ considerably. These discrepancies require an explanation. One reason could be that the totality of the data gradually became more and more positive as more RCTs were published. Yet, Table 1 shows that neither the number of primary studies nor the total sample size of the systematic reviews increased in a linear fashion. Furthermore, there is no clear trend to suggest that the primary studies became more and more positive over time (data not shown). What is striking, however, is that those systematic reviews arriving at positive conclusions (Sun et al., 2008; Wang, Sun, et al., 2008; Wang, Qi, et al., 2008; Xiong et al., 2009; Zhang et al., 2010; Zhong et al., 2008) all originated from China and, with the exception of one (Wang, Qi, et al., 2008), they all relied heavily on primary studies from China. Crucially, these reviews also tended to exclude non-chinese RCTs: in 2007, at least four non-chinese RCTs were available (Leo & Ligot, 2007), yet these studies were excluded from the systematic reviews that most heavily relied on Chinese studies (Sun et al., 2008; Wang, Sun, et al., 2008; Zhang et al., 2010). Vickers et al. were the first who pointed out that virtually 100% of all acupuncture trials originating from China are positive, that is, suggest that acupuncture is effective (Vickers, Goyal, Harland, & Rees, 1998). This
410 Evaluation & the Health Professions 34(4) Table 2. Information of the Direction of Results Design Number of Primary Studies (Total Number ¼ 71) þ + AT Versus Sham AT AT Versus Drug AT Versus No-Treatment or Wait-List; AT þ B Versus B study a High 9 6 1 2 5 2 2 Moderate 34 18 13 3 4 27 3 Low 28 13 11 4 3 25 0 Origin of study China 64 31 24 6 5 52 5 United States 3 3 0 0 3 0 0 Germany 2 2 0 0 2 0 0 United Kingdom 1 0 0 1 1 0 0 Design AT versus sham AT 12 7 2 3 AT versus drug 52 27 21 4 AT versus notreatment or wait-list; ATþB versus B 5 3 2 0 Note. AT¼ acupuncture; þ¼clearly positive; ¼clearly negative; + ¼ equivocal. a Based on Jadad score. High means 4, moderate ¼ 3, and low 2. finding was later confirmed by other researchers (Tang, Zhan, & Ernst, 1999). The current analysis suggests a similar phenomenon: of the 64 primary studies from China, only 6 were negative. Our analysis suggests that also systematic reviews originating from China are far more likely to be positive than reviews from other countries. In principle, the discrepancy between results from China and elsewhere could be due to false negative results of research originating from outside China. Much more likely, however, it is due to false positive results of research from China. As far as we know, the reason for this phenomenon is unknown. It casts serious doubts about the reliability of Chinese acupuncture trials and could provide the key to understanding the discrepancies in the reviews evaluated here: if a systematic review includes numerous false positive primary studies, its overall conclusion is likely to be false positive as well. We might therefore be faced with a conundrum. On one hand, not considering acupuncture trials or reviews from China (where acupuncture is
Ernst et al. 411 practiced more frequently than elsewhere) could mean neglecting crucial information. On the other hand, including them might risk generating a misleading impression about the therapeutic value of acupuncture. It would seem prudent to rely on those systematic reviews that include all available data are critical in their evaluations and cautious in their conclusions (Leo & Ligot, 2007; Mukaino et al., 2005; Smith et al., 2010). In essence, this means that, based on such systematic reviews, the evidence is consistent with acupuncture effects in depression being indistinguishable from placebo effects. Declaration of Conflicting Interests The author(s) declared no conflicts of interest with respect to the authorship and/or publication of this article. Funding The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: MSL and TY Choi were funded by KIOM (K10251). References Ernst, E., Rand, J. I., & Stevinson, C. (1998). Complementary therapies for depression. Archives of General Psychiatry, 55, 1026-1032. Leo, R. J., & Ligot, J. S. Jr. (2007). A systematic review of randomized controlled trials of acupuncture in the treatment of depression. Journal of Affective Disorders, 97, 13-22. Mukaino, Y., Park, J., White, A., & Ernst, E. (2005). The effectiveness of acupuncture for depression: A systematic review of randomised controlled trials. Acupuncture in Medicine, 23, 70-76. Oxman, A. D., & Guyatt, G. H. (1991). Validation of an index of the quality of review articles. Journal of Clinical Epidemiology, 44, 1271-1278. Smith, C. A., Hay, P. P. J., & MacPherson, H. (2010). Acupuncture for depression. Cochrane Database of Systematic Reviews, Issue 1, Art. No.: CD004046. doi: 10.1002/14651858.CD004046.pub3. Sun, Y. L., Chen, S. B., Gao, Y., & Xiong, J. (2008). Acupuncture versus western medicine for depression in China: A systematic review. Chinese Journal of Evidence-Based Medicine, 8, 340-345. Tang, J. L., Zhan, S. Y., & Ernst, E. (1999). Review of randomised controlled trials of traditional Chinese medicine. British Medical Journal, 319, 160-161.
412 Evaluation & the Health Professions 34(4) Vickers, A., Goyal, N., Harland, R., & Rees, R. (1998). Do certain countries produce only positive results A systematic review of controlled trials. Controlled Clinical Trials, 19, 159-166. Wang, L., Sun, D. W., Zou, W., & Zhang, J. Y. (2008). Systematic evaluation of therapeutic effect and safety of acupuncture for treatment of depression. Zhongguo Zhenjiu, 28, 381-386. Wang, H., Qi, H., Wang, B. S., Cui, Y. Y., Zhu, L., Rong, Z. X., & Chen, H. Z. (2008). Is acupuncture beneficial in depression: a meta-analysis of 8 randomized controlled trials? Journal of Affective Disorders, 111, 125-134. Xiong, J., Du, Y., Liu, J., Lin, X., Sun, P., Xiao, L., & Chen, Y. W. (2009). Acupuncture versus western medicine for depression neurosis: a systematic review. Chinese Journal of Evidence-Based Medicine, 9, 969-975. Zhang, Z. J., Chen, H. Y, Yip, K. C., Ng, R., & Wong, V. T. (2009). The effectiveness and safety of acupuncture therapy in depressive disorders: Systematic review and meta-analysis. Journal of Affective Disorders, 124, 9-21. doi: 10.1016/j.jad.2009.07.005. Zhong, B. L., Huang, Y. Q., & Li, H. J. (2008). The effectiveness and safety of acupuncture for depression: A systematic assessment. Chinese Mental Health Journal, 22, 641-647.