REPRODUCTIVE ENDOCRINOLOGY

Transcription

1 FERTILITY AND STERILITY VOL. 74, NO. 2, AUGUST 2000 Copyright 2000 American Society for Reproductive Medicine Published by Elsevier Science Inc. Printed on acid-free paper in U.S.A. REPRODUCTIVE ENDOCRINOLOGY The alternating-sequence design (or multiple-period crossover) trial for evaluating treatment efficacy in infertility Geoffrey R. Norman, Ph.D., a and Salim Daya, M.B., Ch.B., M.Sc. a,b McMaster University, Hamilton, Ontario, Canada Objective: To determine whether a constant-sequence or an alternating-sequence design is better for the evaluation of infertility treatment efficacy when multiple cycles of treatment are undertaken. Design: A simulation exercise using analytical methods. Setting: University medical center. Patient(s): A hypothetical, heterogeneous population of infertile patients participating in a randomized trial comparing an experimental treatment, with effectiveness of 2.0, to no treatment. Intervention(s): Comparison of a constant-sequence design in which the subject receives the same intervention or the alternating-sequence design in which experimental and control treatments are crossed over after each successive cycle. Main Outcome Measure(s): Relative risks of pregnancy per cycle and overall after a maximum of five cycles of treatment. Result(s): With both designs, the pregnancy rates in experimental and control groups showed a consistent decrease with each successive cycle. The overall effectiveness in the constant-sequence design was underestimated at 1.83, whereas in the alternating-sequence design it was overestimated at However, by restricting the analysis in the latter design only to the odd-numbered cycles, the relative risk was precisely correct at Conclusion(s): When multiple cycles of treatment are undertaken to evaluate the efficacy of infertility therapy, the alternating-sequence design with restriction of the analysis to only the odd-numbered treatment cycles provides an unbiased estimation of the treatment effect. (Fertil Steril 2000;74: by American Society for Reproductive Medicine.) Key Words: Infertility, efficacy, randomized controlled trial, constant-sequence design, alternating-sequence design Received October 20, 1999; revised and accepted January 31, Reprint requests: Salim Daya, M.B., Ch.B., M.Sc., Department of Obstetrics and Gynaecology, McMaster University, 1200 Main Street West, Hamilton, Ontario, Canada L8N 3Z5 (FAX: ; dayas@fhs.csu.mcmaster.ca). a Department of Clinical Epidemiology and Biostatistics. b Department of Obstetrics and Gynaecology /00/$20.00 PII S (00) The rapid development in assisted reproductive technology has provided infertile couples with a variety of therapeutic options to achieve pregnancy. Together with this change has been a shift in the focus of reporting success rates of treatment from a per-patient basis to a percycle basis. This change has made the evaluation of treatment efficacy more complicated because of variability in the numbers of cycles of treatment per patient and in the length of time patients may have to wait between successive cycles of treatment. In the past, the pregnancy rates with treatments, such as tuboplasty, donor insemination, and ovulation induction, were compared with those in a control group (often receiving no treatment) at a specified time period (e.g., 1 year after initiating treatment). With this approach, patients in the experimental group were exposed to a single type of intervention. Today, the situation is different because the approach is to offer several cycles of treatment with assisted reproductive technology (ART), the nature of which may vary from cycle to cycle, and to evaluate the outcome per treatment cycle. In the one-cycle, randomized, paralleldesign trial, the experimental treatment is administered in one cycle in one group of patients, and the control treatment is administered in one cycle in a second group of patients. This design permits comparison of pregnancy rates between patients and is the best design for evaluating treatment efficacy, but a large sample size is usually required because, in general, 319

2 small differences (e.g., 5% 10%) in pregnancy rates are expected. Furthermore, compliance may be more difficult to achieve, especially in patients allocated to the control group. In contrast, the two-period (or two-cycle) crossover design is becoming more popular among clinical investigators and may be more appealing to patients because they have the opportunity to receive the experimental treatment in a second cycle if they had received the control treatment in the first cycle. In this way, all patients would have access to the experimental treatment, and the sample size would be much smaller than in the parallel design because a within-patient comparison is undertaken. However, a major problem with this design is that women who become pregnant in the first period will be classified as dropouts from the second period, thereby not permitting a within-patient comparison (1 6). In addition, the likelihood of becoming pregnant may not be constant from one period to the next because women who fail to conceive in the first period may have a lower probability of success in the second period. Consequently, by pooling the data obtained over the two study periods, a larger estimate of the effect of treatment is obtained than that obtained with a one-cycle parallel design trial (7). This issue of possible bias resulting from study design has been explored further with computer simulation and extending the number of cycles per patient to six (8). With use of a constant-treatment sequence design (in which a parallel group design is used but patients undertake repeated cycles of the same treatment within each group), the results were observed to be about the same as those with an alternatingsequence design (in which experimental and control treatments are crossed over after each successive cycle). Although the pregnancy rates in the two designs using the computer simulation seem close, it is difficult to know from the simulation whether they are the same or different. The computed odds ratios (ORs) in the cycles of the constant-sequence arm ranged from 1.76 to 2.25, whereas in the alternating-sequence arm, the ORs ranged from 2.18 to Thus, there was considerable variability in the results, such that small but systematic differences may have been obscured. This variability is a natural consequence of the statistical simulation technique, in which, even with a starting set of 2,000 observations, individual data may vary considerably, simply from sampling variation. An alternative approach is to use a somewhat simpler form of simulation in which results are computed directly from assumed probability distributions. The objective of this study was to undertake such a simulation exercise, with parameters chosen so that real differences would emerge, but with a form sufficiently simplified that analytical methods could be used to solve the question of whether a constantsequence design or an alternating-sequence design is better for the evaluation of efficacy of infertility therapy when multiple cycles of treatment are undertaken. MATERIALS AND METHODS We conducted an analytical simulation starting with the assumption that we are dealing with an experimental treatment with effectiveness of 2.0. The heterogeneity of the sample was modeled in a simple manner by assuming that each cohort consisted of two subpopulations: 20% of the sample consisted of fertile women who have a spontaneous fecundity rate of 40% per cycle, and 80% of the sample consisted of relatively less fertile couples with a spontaneous fecundity rate of 10% per cycle. Thus, based on the effectiveness of 2.0, the fecundity rates in the experimental arm of the model would be 80% (from the fertile 20% of the sample) and 20% (from the remaining, relatively less fertile, 80% of the sample). In addition, for convenience, the starting sample size in each arm was set at 1,000 couples who will each undergo up to a maximum of five cycles of treatment. Although the fecundity rates were chosen to be unrealistically high to make the effects clear, the issue of varying the control group pregnancy rates was also evaluated with use of algebra (as indicated below). The analysis was directed at determining the estimated effectiveness of the experimental treatment over successive cycles to identify whether the presumed bias of both the constant-sequence and alternating-sequence designs are present. In this analysis, the effectiveness of treatment was assessed with use of the relative risk (RR) (i.e., risk of pregnancy in the experimental group/risk of pregnancy in the control group) rather than the OR, because the RR is independent of prevalence, whereas the OR is affected by the overall prevalence and would be expected to become smaller with successive cycles. The results were computed with a handheld calculator with use of a simple approach. For the first treatment cycle, the number of pregnancies was calculated in the two subgroups ( in the control group and in the experimental group). These numbers were then subtracted from the starting population sizes to give the new populations for the second cycle (i.e., 120 and 720, respectively, in the control group, and 40 and 640, respectively, in the experimental group). For the constant-sequence design study, the number of pregnancies in the new cohorts were then calculated as ( ) 120 for the control group and ( ) 160 for the experimental group. In the alternating-sequence design study, the populations were crossed over (i.e., in the second cycle the experimental group now receives the control treatment, whereas the control group now receives the experimental treatment). The calculation was then repeated so that the number of pregnancies in the new experimental group was now ( ) 96, and for the new control group it was ( ) 80. This process was iterated for five cycles. For each design, the RR for pregnancy was then calculated per cycle and overall. 320 Norman and Daya The alternating-sequence design trial Vol. 74, No. 2, August 2000

3 FIGURE 1 Relative risk of pregnancy using the constant-sequence design trial. The assumption is that each sample is made up of 20% with a spontaneous fecundity of 40% per cycle and 80% with a spontaneous fecundity of 10% per cycle, and treatment has an effectiveness of 2.0. RR (overall) The results are shown in Figure 1 for the constant-sequence design and in Figure 2 for the alternating-sequence design. In the former design, the proportion of women with high fertility decreases in both arms, but at a much higher rate in the experimental arm, so that after five cycles, although approximately 26 patients remain in the control arm, almost none remains in the experimental arm. Consequently, the pregnancy rate in each group shows a consistent decrease from the first to the last cycle. However, as the numbers of couples in the experimental group diminish, the pregnancy rate asymptotes to 0.20, whereas the rate in the control group continues to fall (Table 1). As a result, the effectiveness (i.e., pregnancy rate in experimental group/pregnancy rate in control group) is at a minimum with the third cycle and increases slightly thereafter. The overall computed effectiveness is 1.83, somewhat smaller than the true value of The alternating-sequence design study shows a different picture. As expected, in the second cycle, the experimental group of patients now contains a relatively higher proportion of fertile couples, so that the effectiveness of the experimental treatment is overestimated at 2.42 (Table 2). However, in the third cycle, the crossover has removed the excess of fertile women from the new control group, so the proportion of fertile couples is exactly the same in both groups. The calculated effectiveness is now precisely correct at In cycle four, the new experimental group again has a relative excess of fertile women, and the calculated effectiveness is high at In cycle five, however, the numbers of fertile and infertile couples in the two groups again are exactly the same, and the calculated effectiveness, once more, is precisely correct at Overall, the effectiveness of the treatment in the alternating-sequence design trial is It is important to note that these results are based on two basic assumptions only: [1] There is no carryover effect from one cycle to the next. That is, the effect of the drug acts only on the cycle in which it is taken, and does not continue into subsequent cycles. [2] The distribution of fertility takes a particularly simple form, consisting of two subpopulations with different fixed fertility rates. Although we have used rates of spontaneous fertility (10% and 40%) that are unrealistically high and a drug FIGURE 2 Relative risk of pregnancy using the alternating-sequence design trial. The assumption is that each sample is made up of 20% with a spontaneous fecundity of 40% per cycle and 80% with a spontaneous fecundity of 10% per cycle, and treatment has an effectiveness of 2.0. RR (overall) RESULTS FERTILITY & STERILITY 321

4 TABLE 1 Pregnancy rate per cycle in the constant-sequence design trial. Cycle number Experimental group Pregnancy risk a Control group Relative risk 1 320/1000 (0.320) 160/1000 (0.160) /680 (0.235) 120/840 (0.143) /520 (0.209) 93.6/720 (0.130) /411.2 (0.202) 75.6/626.4 (0.121) /328 (0.200) 62.8/550.8 (0.114) 1.76 Overall 737.8/ (0.251) 512/ (0.137) 1.83 a Pregnancy risk refers to the number of subjects becoming pregnant out of the total number receiving the experimental or control interventions, and the proportions are indicated in parentheses. efficacy that perhaps is similarly unrealistic, the findings of the simulation related to the unbiased nature of the odd-cycle alternating sequence design do not depend on the values chosen. The high values are used simply to make the results more obvious; the results are the same with any arbitrary set of starting parameters. Furthermore, the assumptions of a constant drug efficacy and a simple distribution of fertility are not necessary. In the Appendix, we show algebraically that, for any distribution of fertility p(f), where F is a number between 0 and 1 and p(f) expresses the proportion of women in a sample with a particular level of fertility, and for any value of drug efficacy, (F), where the efficacy is a function of fertility, the outcome rates in the odd cycles in an alternating sequence are unbiased. In particular, we might presume clinically that the efficacy of the drug would be reduced in high fertility women; this algebraic extension of the basic concept demonstrates that the results will hold true regardless of the relationship between efficacy and fertility. One other issue is whether the possibility of bias justifies rejecting up to half the data. To assess this issue, we recalculated the results with use of the alternating-sequence design after assuming more conservative estimates of fertility (i.e., 2.5% and 10%) in the two subgroups. Under these circumstances, the odd-numbered cycles continued to provide unbiased estimates of efficacy; the even-numbered cycles yielded estimates that were biased upward, but to a lesser degree 2.10 for cycle 2 and 2.05 for cycle 4. The only remaining assumption is the one of lack of carryover effect. If there was a carryover effect, such that the effect of the drug continued beyond the cycle in which it was administered, then the effect would be to increase fertility in the control group at each cycle, resulting in a conservative estimate of treatment effectiveness. Returning to the original, simplified, model, we assumed a carryover effect of 20% (i.e., the fertility rate in the control subsamples would increase from 10% and 40% to 12% and 48%, respectively) and recalculated the results. The calculation yielded an estimate of 1.99 for all cycles and 1.82 for the odd cycles. It is tempting to assume that, in situations in which it is presumed there is a carryover effect, one should use an alternating-sequence design and include all cycles in the analysis. However, the computed efficacy of 1.99 in this situation results from two competing biases that will not cancel out in all circumstances. DISCUSSION This small analytical simulation has led to a remarkable conclusion. When multiple cycles of treatment are undertaken to evaluate the efficacy of infertility therapy, neither the constant-sequence design (in which the experimental group receives the same intervention in successive cycles) nor the alternating-sequence design (in which experimental and control interventions are crossed over at the end of each successive cycle) provides the correct estimate of the true treatment effect. Both designs suffer from systematic bias due to confounding of treatment effect and dropouts that result from pregnancy. However, the use of the alternatingsequence design with restriction of the analysis to only the odd-numbered treatment cycles provides an unbiased estimation of the treatment effect. Although it is tempting to presume that this unexpected finding may represent an idiosyncracy of the simple model, there is every reason to believe that the findings are generalizable to situations in which more than two cycles of treatment are undertaken. The simple reason for this belief is that, in general, after two cycles the increased pregnancy rate of the experimental treatment has been applied to both experimental and control groups so that the distributions are equivalent regarding fertility potential. The analysis we performed in the Appendix shows that the finding is robust over TABLE 2 Pregnancy rate per cycle in the alternating-sequence design trial. Cycle number Experimental group Pregnancy risk a Control group Relative risk 1 320/1000 (0.320) 160/1000 (0.160) /840 (0.286) 80/680 (0.118) /600 (0.224) 67.2/600 (0.112) /532.8 (0.216) 48/465.6 (0.103) /417.6 (0.204) 42.7/417.6 (0.102) 2.00 Overall 894.5/ (0.260) 397.9/ (0.126) 2.06 a Pregnancy risk refers to the number of subjects becoming pregnant out of the total number receiving the experimental or control interventions, and the proportions are indicated in parentheses. 322 Norman and Daya The alternating-sequence design trial Vol. 74, No. 2, August 2000

5 any estimate of the distribution of fertility and treatment effectiveness. Moreover, the results we obtained are remarkably similar to those of Cohlen s more elaborate model (8). In addition, their computer simulation model shows exactly the same pattern we predicted, with even-numbered cycles producing ORs that are systematically higher than those of odd-numbered cycles. In fact, the effectiveness calculated from the odd-numbered cycles of their model average remarkably close to the true value (i.e., cycle 1, 20/ ; cycle 3, 17.9/ ; and cycle 5, 16.3/ ), just as was the case with our simplified model. Neither the constant-sequence nor the alternating-sequence design is immune to the potential for bias from differential numbers of dropout. In both cases, women who are most fertile will be the ones most likely to drop out (i.e., become pregnant). Consequently, the population at each successive cycle constitutes a progressively more infertile group. Moreover, when the treatment is effective, there will be a systematic distortion of the two groups. Thus, with effective treatment, the relatively more fertile patients in the experimental arm in the constant-sequence design will drop out at a higher rate than those in the control arm so that over successive cycles, the treatment will seem to become progressively less effective. This effect of decreasing OR in a linear fashion over successive cycles was also observed in the computer simulation study (8). The net result of this phenomenon is that the constant-sequence design study may underestimate the true therapeutic effect. In the present study, the final estimated RR was 1.83 instead of 2.00, and in the computer simulation study the final estimated OR was 2.05 instead of 2.25 (8). Conversely, in the alternating-sequence design trial, patients who have not conceived in the first cycle are now crossed over into the alternative group. Thus, those who were in the control group are now in the experimental group and vice versa. A potential problem in the opposite direction may now be encountered. If therapy has been effective in the first cycle, the new experimental group will now consist of couples that are systematically more fertile than those in the new control group. Consequently, treatment would appear more effective than it is in reality. Therefore, the alternatingsequence design trial will produce results that may overestimate the true treatment effect. The situation is more complicated than that of the constant-sequence design trial, because the effective treatment is now acting on the subgroup with relatively more fertile women thereby producing, at the end of the second cycle, two groups that are somewhat more similar. This effect is also seen in the computer simulation study in which the successive ORs for the even-numbered cycles were 2.40, 2.36, and 2.36 compared to 2.25, 2.26, 2.18, and 2.30 in the odd-numbered cycles (8). The one caveat to the use of the alternating-sequence design is that, in situations in which there may be carryover effects, the consequence will be to increase fertility in the control group and thereby underestimate the treatment effect. The finding of this study is of particular importance in present-day infertility research because of the increasing emphasis that is being placed on the two-period crossover design trial to evaluate therapy. It can be seen clearly from Table 2 that because the second cycle produces a larger estimate of the treatment effect, the overall conclusion at the end of the two periods (or cycles) will be an overestimation of the true treatment effect, i.e., the estimate in period 1 (or cycle 1) is 2.00 (as expected), but in period 2 (or cycle 2) it is much higher at 2.42, resulting in an overall estimate at the end of the two-period crossover study of A more reliable approach is to conduct a one-period trial in which patients are randomly allocated to receive either treatment or control intervention in one cycle. Such a parallel group design trial, with sufficient power, is the best design to evaluate treatment efficacy and would provide an estimate of treatment effect that is closer to truth. However, it requires a much larger sample size than with the crossover design, and half of the patients would have to be in the control (or nonintervention arm) of the trial. Hence, it is not as popular with patients as is the two-period crossover design, which ensures that patients allocated to the control arm in the first period will receive the experimental treatment in the second period. The objective of obtaining an accurate estimate of the effect of treatment, but also allowing all subjects to have the opportunity to receive the experimental treatment in at least one cycle, can now be achieved with the alternating-sequence design trial. The proviso is that the trial should run for at least three cycles and all data from the even-numbered cycles would have to be excluded from the analysis, which would be restricted only to the odd-numbered cycles. Because up to half of the data have to be discarded with this approach, the design has obvious implications for calculating sample size, which is now expressed in cycles and not in persons. Of course, with lower fecundity rates, the magnitude of the bias is reduced, and it may be that individual investigators may choose to accept the possibility of bias in return for an increase in statistical power. This strategy of using the alternating-sequence design trial may have implications beyond infertility trials. In situations where there is a differential susceptibility in the population and where the outcome leads to a dropout from the trial, such as would arise from either cure or death, then effective treatments inevitably will be applied to progressively more resistant subpopulations as time unfolds. Under these circumstances, it may be wise to induce a period effect, so that the treatment can be applied equally to both cohorts, thereby maintaining similar characteristics in the experimental and control samples over time. FERTILITY & STERILITY 323

6 References 1. Daya S. Is there a place for the crossover design in infertility trials? Fertil Steril 1993;59: Lois TA, Lavori PW, Bailar JC, Polansky M. Crossover and selfcontrolled designs in clinical research. N Engl J Med 1984;310: Hills M, Armitage P. Two-period crossover clinical trial. Br J Pharmacol 1979;82: Petrie A. The crossover design. In: Tygstrup N, Lachin JM, Juhl E, editors. The randomized clinical trial and therapeutic decisions. New York: Marcel Dekker Inc. 1982: Senn S. Cross-over trials in clinical research. Chichester, England: John Wiley and Sons Ltd. 1993:8. 6. Daya S. Differences between crossover and parallel study designs debate? Fertil Steril 1999;71: Khan K, Daya S, Collins JA, Walter SD. Empirical evidence of bias in infertility research: overestimation of treatment effect in crossover trials using pregnancy as the outcome measure. Fertil Steril 1996;65: Cohlen BJ, le Velde ER, Looman CWN, Eijckemans R, Habbema JDF. Crossover or parallel design in infertility trials? The discussion continues. Fertil Steril 1998;70:40 5. APPENDIX Let p(f) be the probability distribution of women as a function of fertility (F), where F ranges from 0 to 1. So, for a sample of size N, the number of women at a given fertility, F is equal to Np(F). Furthermore, in the most general case, the effectiveness of the drug,, will also be related to fertility, so is expressed as (F). So, at each cycle, the number of pregnancies in the control group will be F the number of patients at risk, and in the experimental group will be F (F) the number at risk. At the conclusion of each cycle, then, the number remaining at risk (who will then be crossed over at the next cycle) is (1 F) the number at risk in the control group, and (1 F (F)) the number at risk in the experimental group. We can now compute the number of pregnancies at each cycle for the crossover sequence, as shown in Appendix Table 1. Because the number of patients at fertility F is exactly the same in both groups at the beginning of cycle 3, we have proved that the effectiveness computed from odd-numbered cycles is an unbiased estimate, in the general case of an arbitrary distribution of fertility and a drug effectiveness that is a function of fertility. APPENDIX TABLE 1 Theoretical calculation of the number of pregnancies and number of exposed patients for three cycles of a crossover sequence, for the general case of an arbitrary distribution of fertility and effectiveness related to fertility. Control Treatment Cycle 1 No. at risk N 1c Np(F) N 1t Np(F) No. pregnant F N 1c F N p(f) F (F) N 1t F (F) N p(f) No. at risk at end of cycle (1 F) N 1c (1 F)N p(f) (1 F (F)) N 1t (1 F (F)) N p(f) Cycle 2 No. at risk N 2c (1 F (F)) N p(f) N 2t (1 F) N p(f) No. pregnant F N 2c F(1 F (F)) N p(f) F (F)N 2t F (F)(1 F) N p(f) No. at risk at end of cycle (1 F) N 2c (1 F)(1 F (F)) N p(f) (1 F (F)) N 2t (1 F (F)(1 F) N p(f) Cycle 3 No. at risk N 3c (1 F)(1 F (F)) N p(f) N 3t (1 F (F)(1 F) N p(f) 324 Norman and Daya The alternating-sequence design trial Vol. 74, No. 2, August 2000