How "Blind" Are Double-Blind Placebo-Controlled Trials of Benzodiazepine Hypnotics?

Sleep, 18(4):240-245 1995 American Sleep Disorders Association and Sleep Research Society How "Blind" Are Double-Blind Placebo-Controlled Trials of Benzodiazepine Hypnotics? Charles M. Morin, Cheryl Colecchi, Doug Brink, Manual Astruc, James Mercer and Stephanie Remsberg Medical College of Virginia/Virginia Commonwealth University, Richmond, Virginia, U.S.A. Summary: This study examined the accuracy of insomnia patients and their treating physicians in whether an active hypnotic drug or a placebo was given in treatment. Forty older adults with primary insomnia were randomly assigned to either an active (temazepam) or a placebo condition using a double-blind strategy. Ratings of treatment conditions were obtained at I week (early treatment), 4 weeks (midtreatment), and 8 weeks (late treatment). Patients were able to accurately discriminate (beyond chance levels) between the active and placebo medications at the early (76.9% accuracy) and late treatment assessment timepoints (78.1% accuracy), but not at mid treatment (51.5% accuracy). Therapists, however, were able to make accurate discriminations at the late treatment assessment timepoint only (80% accuracy); early (69.2% accuracy) and midtreatment (47.2% accuracy) s did not exceed chance levels. Patients who had used hypnotic drugs prior to this trial were more accurate in their judgments of treatment conditions than those without prior exposure. The findings raise an important issue about the internal validity of the double-blind strategy, which may in fact be only a single-blind procedure. The double-blind procedure is the standard method used in psychopharmacologic studies to control for nonspecific effects, such as those caused by patient expectations, experimenter bias (e.g. instructional demands) and treatment credibility (1,2). It is generally assumed that the effects of these potentially confounding variables will be minimized or eliminated by withholding the nature ofthe treatment condition from the patient and therapist (i.e. whether an active or placebo is given). Furthermore, this strategy presumes that both the patient and therapist are unable to distinguish on their own the type of treatment given. This presumption, however, has never been evaluated empirically in controlled trials of hypnotic drugs, raising an important issue about the internal validity ofthe doubleblind strategy. Several controlled studies of other psychoactive drugs recently have questioned the validity of the doubleblind procedure (3-7). For instance, in clinical trials of lithium, imipramine and nicotine drugs, participants were able to determine beyond chance levels whether the medication received was active or a pla- Accepted for publication November 1994. Address correspondence and reprint requests to Charles M. Morin, Ph.D., Universite Laval, Ecole de Psychologie, Ste-Foy, Quebec, Canada GlK 7P4. 240 cebo. Because the s were obtained at the end of treatment, it was impossible to determine whether the s were influenced by treatment outcome or vice versa. Margraf et al. (8) addressed this problem in their study of patients with panic disorders by having patients and therapists rate halfway through treatment (at 4 weeks) whether alprazolam, imipramine or a placebo was given. It was found that both patients and therapists were accurate in their s halfway through treatment. However, the authors cautioned that treatment effects might have already been evident this early in treatment, thereby contributing to the accuracy of s. They pointed out the need for future studies to assess patients' and physicians' s about the treatment condition as early as the 1 st week of treatment. The assessment of the double-blind method early on in treatment is especially important in insomnia research because, unlike some other psychotropic medications (e.g. antidepressant) whose therapeutic effects may be delayed, hypnotic drugs have more immediate effects that may also dissipate over time (9,10). Also, because sleep improvements as well as side effects may change over time, it is important to evaluate the double-blind procedure at repeated intervals during the course of treatment. In addition to clinical response and side effects, other factors (e.g. prior experience with

HOW "BLIND" ARE DOUBLE-BLIND TRIALS? 241 -, hypnotic drugs) that may mediate patients' accuracy in distinguishing between a placebo and an active sleep medication should be evaluated_ The purpose of the present study was to examine whether insomnia patients and their physicians were really "blind" to treatment conditions involving an active sleep medication or a placebo. To determine the extent to which treatment outcomes impact the individual's s, patients and therapists were asked to determine whether an active or inactive medication was given at three assessment points during the course of an 8-week clinical trial. Due to the more immediate therapeutic effects and side effects of hypnotic medications, it was expected that patients and physicians would be able to determine whether they received an active or a placebo drug as early as after the 1st week of treatment. Subjects METHODS Data collection was part of a larger study comparing the clinical efficacy of cognitive-behavior therapy and pharmacotherapy for late-life insomnia (11). Participants were community-residing elderly who responded to newspaper advertisements. They were initially screened over the phone. After this initial screening, each participant underwent a multistep evaluation: (a) a clinical interview with a board-certified sleep specialist, (b) a physical examination, (c) a psychological evaluation and (d) 3 consecutive nights of polysomnography. All participants met criteria for primary insomnia (12), which was mostly of the psychophysiological subtype (13). Minimal entry criteria were that subjects reported chronic (i.e. greater than a 6-month duration) difficulties maintaining sleep, which was defined as wake after sleep onset greater than 30 minutes per night for more than 3 nights per week. Subjects were excluded if there was evidence of significant sleep apnea (respiratory disturbance index> 15) or periodic limb movements (PLM) during sleep (PLM with arousal > 15). All subjects were free from psychotropic drugs for at least 2 weeks prior to entering the study. A total of 78 patients participated in the study; patients were randomly assigned to one of the four following conditions: 18 to cognitive-behavior therapy, 20 to pharmacotherapy (temazepam), 20 to combined cognitivebehavior therapy and pharmacotherapy, and 20 to a drug placebo. The present study considers only patients in the active and placebo medication conditions. Of the 40 participants, 16 were male and 24 were female. Their mean age was 65 years (SD = 6.7 years) and their average education level was 14.9 years (SD = 2.5 years). They were predominantly married (67.5%). Fifty percent were retired, and 37.5% were employed (10% were homemakers and 2.5% unemployed). The average duration of insomnia was 16.5 years (SD = 15.5 years). Thirty-one (78%) of the 40 subjects had previously used prescribed (n = 25) or over-the-counter sleep medications (n = 6); nine subjects had never used sleep aids. Of those who had used a prescribed hypnotic medications four had specifically used temazepam. Procedures Subjects were randomly assigned to the active or placebo condition. Treatment was provided in a double-blind fashion such that neither the patients nor the therapists were informed of whether an active or placebo drug was involved for any given patient. The patients were informed about the four treatment conditions being compared and were told that their chance of being assigned to anyone condition was approximately 25%. Pharmacotherapy consisted of temazepam. Patients were all started on 7.5 mg and medication could be increased, based on the patient's subjective response, to a maximum of 30 mg. The dosage could be decreased, based on side effects, to a minimum dosage of7.5 mg. It was recommended that the patient use the medication 2-3 nights per week, but it was made available for nightly use as needed. The placebo medication consisted of an inert lactose solution. Both active and placebo drugs were provided in identical gelatin capsules filled with an equivalent amount of powder. These capsules were dispensed by the Investigational Drug Pharmacy. Dosages of up to 22.5 mg oftemazepam were included in one capsule. Only those subjects taking 30 mg were given two capsules. Subjects in the placebo condition were given the same feedback regarding dosage of medication and an identical opportunity as those in the active medication condition to increase their dosage to a maximum of 30 mg or two capsules. All treatments lasted 8 weeks. Subjects in both treatment conditions underwent a prescreening psychiatric and medical evaluation by a thirdyear psychiatry resident, who then continued to see each subject once a week for a 20-30-minute session devoted to reviewing treatment progress, side effects and insomnia symptoms. Urine drug screens for benzodiazepines were conducted at baseline to ensure that subjects were free from these agents and at late treatment to ensure compliance with prescribed medication regimen. At the end of the 8 weeks (i.e. 1 week after the eighth therapy session), all subjects began a 1-2 week tapering schedule. The dosage was gradually decreased to the next lowest level for 2 or more nights. This pattern was repeated until the lowest dosage (7.5 mg) was Sleep. Vol. 18. No.4. 1995

242 C. MORIN ET AL. reached. Medication was then discontinued. For example, a subject using 30 mg at the end of treatment would be tapered to 22.5 mg for 2-3 nights, 15 mg for 2-3 nights and 7.5 mg for the last 2-3 nights before discontinuation. This tapering schedule was followed in an identical fashion for subjects in both the active and placebo conditions. Measures Participants and therapists were asked to independently judge whether an active medication or a placebo had been given on three separate occasions: after the 1 st week (early treatment), the 4th week (midtreatment) and the 8th and final week (late treatment). The late treatment assessment occurred at the last treatment session (i.e. 1 week before the tapering schedule began). The patients were asked to make this determination on a treatment evaluation questionnaire, which included several other items. The question read: "Do you think you are receiving (or for therapists, "this patient is receiving") an active sleep medication or a placebo (inactive substance)? Please make a choice even if you're not sure". Responses were: "(a) Active sleep medication" or "(b) Placebo (inactive substance)". At all three assessment points, both therapists and patients were also asked to rate the certainty of their judgment regarding whether an active medication or placebo had been given by answering the following question: "How certain are you that your answer to question 1 (i.e. active or placebo) is correct"? They were instructed to place a mark on a 100-mm visual analog scale anchored at one end by "Not at all certain" and the other end by "Absolutely certain". An additional on an identical visual analog scale was obtained for perceived side effects. The item asked: "To what extent have you experienced side effects as a result of using this sleep medication"? Responses were converted into a metric measurement of the distance of their mark on the continuum. Finally, patients and their therapists were asked to "Rate the degree of change, if any, in your (this patient's) sleep pattern since you (this patient) first enrolled in this treatment program". Ratings were on a scale of 1 to 5; 1 was equal to "Unchanged or worse", 2 to "Slightly improved", 3 to "Improved", 4 to "Much improved", and 5 to "Very much improved". Patients' forms were secured in a sealed envelope to which the therapists did not have access. RESULTS The following research questions were examined: (a) How accurate are insomnia patients and their therapists in identifying treatment conditions (active vs. Sleep, Vol. 18, No.4, 1995 placebo)? (b) Are subjects on active medication more accurate in identifying their treatment than those receiving placebo? (c) Is accuracy in identifying treatment conditions mediated by treatment response, side effects and prior exposure to hypnotic medications? All categorical variables were analyzed with x 2 tests (with Yates corrections for continuity) and continuous variables were analyzed with either t tests or repeatedmeasures analyses of variance (ANOVAs). When multiple comparisons were computed, Bonferroni corrections were applied to correct for inflated Type 1 error. Preliminary analyses Of the 40 patients enrolled in the study, three patients withdrew from the active drug condition and two from the placebo condition, leaving 35 subjects for whom data were available. Because this project was initiated only after some subjects were already beyond their 1 st week of treatment, data were available for only 26 subjects at the early assessment timepoint. Preliminary analyses revealed that the two groups were equivalent on demographic (e.g. age and gender) and clinical variables (e.g. insomnia severity and duration). The percentage of medicated nights for all subjects was 77% (SD = 17.7%) and the average nightly dosage was 20.3 mg (SD = 3.8 mg). There were no significant group differences (active medication vs. placebo) on either the percentage of medicated nights or the average dosage used per night. All active and placebo subjects used medication a minimum of 3 nights per week for the entire duration of treatment. The number of subjects with and without prior use of hypnotic drugs was also equivalent in both conditions. Patients' accuracy s Patients' s of type of treatment received at the three assessment points are shown in Table 1. Patients' s were 76.9% accurate at early treatment assessment, 51.5% accurate at midtreatment and 78.1 % accurate at late treatment. Chi-square tests (actual vs. hypothesized treatment) revealed that observed values significantly exceeded expected values (a 50% chance of correctly) at early treatment assessment [x 2 (1, n = 26) = 5.4, p < 0.02] and at late treatment assessment [x 2 (1, n = 32) = 8.9, p < 0.005], but not at mid treatment. However, the lack of accuracy at the midpoint appears to be related to the type of treatment received, as only the placebo group was less accurate at this time (38.8%) than chance guessing alone. Physicians' accuracy s Physicians' s are displayed in Table 2. Accuracy rates were 69.2% early in treatment, 47.2% at

HOW "BLIND" ARE DOUBLE-BLIND TRIALS? 243 midtreatment and 80% at late treatment. Those s were significantly different from expected values (i.e. chance levels) at late treatment only [x 2 (l, n = 30) = 8.8, p < 0.005]. Within the active drug condition, accuracy rates were 58.3%, 35.3% and 85.7% at the three assessment points. Accuracy rates within the placebo condition were 78.6% (early), 57.9% (mid) and 75% (late). Patients' and therapists' certainty s The next set of analyses examined levels of certainty in patients' and therapists' s as a function of actual treatment. A repeated-measures ANOV A was conducted on patients' certainty s, using actual treatment condition as the between-group factor and time of assessment as the within-subject factor. There were no significant group or interaction effects. There was, however, a significant effect for time [F(2,40) = 9.1, p < 0.001]. Post hoc comparisons showed that subjects in both conditions became more certain of their s as treatment progressed over time. A repeated-measures ANOV A on therapists' certainty s revealed identical results. Whereas no significant group or interaction effects were obtained, a significant effect for time [F(2,40) = 20.6, p < 0.001] indicated that therapists also were increasingly more confident about their s of treatment conditions as the intervention unfolded. Factors related to accuracy Several sources of information can be used to identify treatment assignments. In this study, we examined whether treatment response, side effects and prior use of hypnotic medication were related to accurate identification of treatment conditions. Treatment response The first set of analyses sought to determine whether treatment responses (i.e. perceived changes in sleep patterns) were related to accuracy of patients' and therapists' s. Subjects receiving the active medication were compared with those receiving the placebo drug. A repeated-measures ANOV A on patients' s of sleep improvements revealed significant effects for group [F(I,22) = 14.72, p < 0.001], time [F(2,44) = 19.98, p < 0.001] and a nearly significant group x time interaction effect [F(2,44) = 2.61, p = 0.08]. Post hoc comparisons indicated that perceived improvements in sleep patterns were, as expected, significantly greater in the active drug condition than in the placebo group (at mid- and late treatment assessment times only), and that patients in both conditions rated their TABLE 1. Identification of condition: patients' responses Correct responses Incorrect responses n (%) [mean (SD)] n (%) [mean (SD)] Early treatment (n = 26) Active 8 (67) 58.8 (32.9) 4 (33) 34.5 (34.6) Placebo 12 (86) 41.6 (24.4) 2 (14) 3.0 (3.0) Midtreatment (n = 33) Active 10 (67) 71.7 (18.2) 5 (33) 36.8 (28.2) Placebo 7 (39) 64.4 (25.2) 11 (61) 57.2 (21.1) Late treatment (n = 32) Active 15 (94) 76.4 (21.3) 1 (6) 7.0 (0.0) Placebo 10 (63) 74.7 (19.2) 6 (37) 58.5 (16.4) sleep patterns as significantly improved over time (all p values < 0.01). A repeated-measures ANOVA on therapists' s of sleep improvements revealed a significant effect for group [F(l,20) = 7.66, p < 0.02] and time [F(2,40) = 12.70, p < 0.001], but not for the group x time interaction. Therapists rated subjects in the active drug condition as significantly more improved than subjects in the placebo condition, and subjects in both conditions as significantly improved from pre-, to mid-, to late treat~ ment (all p values < 0.01). To examine whether the above results could invalidate the overall treatment outcome findings, we compared perceived improvements in the sleep patterns of those who were accurate with those who were inaccurate in their judgments. There were no significant differences for patients or therapists at any of the three assessment points. It is plausible that some individuals who were re~ ceiving an inadequate dose of the active drug, partic~ ularly in the early phase of treatment, thought they were receiving a placebo. These individuals might be viewed as accurate relative to someone receiving an effective dose. To examine this issue we looked at the TABLE 2. Identification of condition: therapists' response Correct responses Incorrect responses n (%) [mean (SD)] n (%) [mean (SD)] Early treatment (n = 26) Active 7 (58) 36.0 (25.6) 5 (42) 37.4 (29.7) Placebo 11 (79) 35.7 (31.9) 3 (21) 73.0 (16.6) Midtreatment (n = 36) Active 6 (35) 76.8 (8.47) 11 (65) 64.7 (22.8) Placebo 11 (58) 76.1 (19.7) 8 (42) 59.1 (18.2) Late treatment (n = 30) Active 12 (86) 80.1 (22.4) 2 (14) 95.5 (4.5) Placebo 12 (75) 77.5 (25.4) 4 (25) 59.0 (33.2) Sleep, Vol. 18, No.4, 1995

244 C. MORIN ET AL. accuracy of a patient's judgment as a function of perceived treatment response within the active drug condition only. Subjects in that condition were divided into two subgroups based on whether they perceived their sleep as improved (self-s of slightly, moderately and much improved) or unimproved (self-s of unchanged or worse). Within the active drug condition, four subjects were inaccurate at the early assessment (two responders and two non responders), five were inaccurate at midtreatment (four responders and one nonresponder), and only one was inaccurate (a nonresponder) at late treatment. When subjects with only slight improvements were considered nonresponders and were combined with those rated as unchanged or worse, a different picture emerged. All but one of the subjects with inaccurate s in the active drug-condition group (i.e. those who thought they were receiving a placebo) were nonresponders. Side effects Subjects receiving the active medication were compared with those receiving the placebo drug for perceived side effects. Repeated-measures ANOVAs on patients' and therapists' s of side effects revealed no significant group, time or group x time interaction effects. Perceived side effects were also compared between subjects who were accurate and those who were inaccurate in identifying their treatment condition, and no significant group differences were obtained for either patients' or therapists' s. Prior use of hypnotic medication To examine the impact of prior exposure to hypnotic medications on accuracy of perception of active versus placebo drugs, patients who previously had used a prescribed hypnotic medication were compared with those who had not. For this analysis, subjects who had used only over-the-counter medications were combined with those who had not used any medications. There was a general trend for subjects with a prior history of hypnotic use to be more accurate than those who had never used hypnotics or those who had used only overthe-counter agents. The percentages of accuracy for the two groups were 80% versus 73% at the early treatment assessment timepoint, 65% versus 31 % at midtreatment and 84% versus 69% at late treatment. Chi-square tests indicated a significant difference in accuracy s between these two groups at midtreatment only [x 2 (1, n = 32) = 5.2, p < 0.05]. Thirteen (65%) of the 20 patients with a prior history of prescribed hypnotic use were accurate, whereas only four (31 %) of 13 without prior use were accurate in judging treatment conditions. DISCUSSION The results showed that insomnia patients were able to accurately determine whether an active benzodiazepine hypnotic or a placebo drug was given as early as 1 week into treatment and at the end of treatment. Patients' s at midtreatment were not different from those expected by chance alone, although this inaccuracy was accounted for primarily by subjects receiving a placebo drug. Prescribing physicians were accurate in their determination of whether an active medication or a placebo was given at the late treatment assessment point only. These findings suggest that the double-blind procedure used in controlled studies of benzodiazepine hypnotics may in fact be only a singleblind method. The temporal fluctuations in accuracy rates over the three assessment phases were somewhat similar for both patients and therapists, at least when the data were collapsed across conditions. Accuracy rates were 77% and 69% at the early treatment assessment timepoint, 52% and 47% at midtreatment, and 78% and 80% at late treatment for patients and therapists, respectively. A different pattern of results seems to emerge, however, when accuracy rates are examined within each treatment condition. For instance, patients in the active drug condition were consistently accurate in identifying their condition across assessment phases, whereas those in the placebo condition showed more fluctuations over time; the midtreatment data were the least accurate for this group. The decline in accuracy s at midtreatment is difficult to explain in light of the increase in certainty s. It is plausible, however, that some nonspecific factors (e.g. therapist attention, expectancy, support) may have temporarily introduced some confounding factors in s of conditions at the midtreatment assessment phase. The therapists were accurate only at the late treatment assessment point, although they appeared to be more accurate at judging a placebo than an active medication in early treatment. Three factors that may account for the failure to maintain blindness were examined in this study: history of hypnotic usage, drug side effects and perceived sleep improvements. First, individuals with previous use of hypnotic medications were more accurate in determining whether they received an active or placebo drug at midtreatment assessment. This finding contrasts with the finding that those on placebo were most inaccurate at midtreatment. Furthermore, the lack of group differences in perceived side effects between active and placebo subjects suggests that this information was not critical in accurately determining treatment conditions. Finally, because perceived improvements in sleep patterns were generally greater in the active Sleep. Vol. 18. No.4. 1995

HOW "BLIND" ARE DOUBLE-BLIND TRIALS? 245 medication group than in the placebo condition, therapeutic effects may have been important in accurately determining treatment conditions. The importance of treatment response as a mediating factor of accuracy is further supported by the active drug-group accuracy s, which were highest at late treatment, i.e. after optimal dosage and, presumably, maximal efficacy had been achieved. An important issue raised by the latter results is whether failure to maintain blindness may invalidate the outcome findings from controlled trials. This issue is not as problematic with therapist s, because those s became more accurate only at the end of treatment; thus, it is unlikely that they had a significant influence on treatment outcome. On the other hand, patients were fairly accurate in determining their treatment condition even in the beginning of treatment. Knowledge of whether an active or placebo drug is involved early in treatment could alter patients' expectations and treatment credibility and potentially invalidate the ultimate outcome findings. However, the finding of no significant differences in perceived sleep improvements between patients who were accurate and those who were inaccurate in their s of treatment suggests that the outcome may still be valid even though blindness was not maintained. The dubious nature of presumed double-blind studies poses a serious challenge to the utilization of this strategy (14). The patient's ability to distinguish between an active medication and a placebo could potentially influence compliance as well as treatment outcome. For instance, if patients believe they are receiving a placebo, they may be less inclined to remain in a clinical trial. Although a potential solution to this problem might be to use an "active placebo", i.e. a substance that would mimic some ofthe side effects of the active medication (15), side effects did not appear to play a significant role in identification of treatment conditions in the present study. Future studies need to explore more systematically the type of information subjects in clinical trials use to determine whether they are receiving an active or placebo drug. Examination of other nonspecific factors (e.g. patients' expectations and acceptance of hypnotic drugs, treatment credibility) is also needed to better understand their mediating role on insomnia treatment outcome (16). Acknowledgements: Preparation of this manuscript was supported by grant R29-MH47020 from the National Institutes of Mental Health, Bethesda, MD. REFERENCES I. K1erman GL. Scientific and ethical considerations in the use of placebo controls in clinical trials in psychopharmacology. Psychopharmacol Bull 1986;22:25-9. 2. White L, Tursky B, Schwartz GE. Placebo: theory, research, and mechanisms. New York: Guilford Press, 1985. 3. Hughes J, Krahn D. Blindness and the validity of the doubleblind procedure. J Clin PsychopharmacoI1985;5: 138-43. 4. Marini JL, Sheard M, Bridges C, Wagner E. An evaluation of the double-blind design in a study comparing lithium carbonate with placebo. Acta Psychiatr Scan 1976;53:343-9. 5. Rabkin JG, Markowith JS, Stewart J, et al. How blind is blind? Assessment of patient and doctor medication guesses in a placebo-controlled trial of imipramine and phenelzine. Psychiatry Res 1986;19:75-86. 6. Rickels K, Hesbacher PT, Weise CC, Gray R, Feldman HS. Pills improvement: a study of placebo response in psychoneurotic outpatients. Psychopharmacologia 1970; 16:318-28. 7. Stallone F, Mendlewicz J, Fieve R. Double-blind procedure: an assessment in a study oflithium prophylaxis. Psychol Med 1975;5: 78-84. 8. Margraf J, Ehlers N, Roth WT, et al. How "blind" are doubleblind studies? J Consult Clin PsychoI1991;59:184-7. 9. Morin CM, Kwentus JA. Behavioral and pharmacological treatments for insomnia. Ann Behav Med 1988; I 0:91-1 00. 10. Gillin JC, Byerley WE The diagnosis and management of insomnia. N Engl J Med 1990;322:351-64. II. Morin CM, Stone J, Maghakian C, Astruc M, Mercer J, Brinks D. Cognitive behavior therapy and pharmacotherapy for latelife insomnia: a placebo-controlled trial. Sleep Res 1993;22:000-000. 12. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, 4th ed. Washington, DC: American Psychiatric Association, 1994. 13. Diagnostic Classification Steering Committee, Thorpy, MJ, chairman. International classification of disorders: diagnostic and coding manual. Rochester, MN: American Sleep Disorders Association, 1990. 14. Fisher S, Greenberg RP, eds. The limits of biological treatments for psychological distress: comparisons with psychotherapy and placebo. Hillsdale, NJ: Erlbaum, 1989. 15. Blumenthal DS, Burke R, Shapiro K. The validity of "identical matching placebos." Arch Gen Psychiatry 1974;31:214-5. 16. Morin CM. Insomnia: psychological assessment and management. New York: Guilford Press, 1993. Sleep, Vol. 18, No.4, 1995