A Noninferiority Margin for Acne Lesion Counts

Transcription

1 STATISTICS 607 Christoph Gerlinger, PhD Senior Director Statistics, Bayer Schering Pharma AG, Berlin, Germany Gerald Städtler, MSc Clinical Statistician, Intendis GmbH, Berlin, Germany Ralf Götzelmann, MSc Senior Statistical Programmer, Bayer Schering Pharma AG, Berlin, Germany Klaus Graupe, MD, PhD Senior Medical Advisor, Intendis GmbH, Berlin, Germany Jan Endrikat, MD, PhD Head Clinical Development and Drug Safety, Bayer Inc., Toronto, Ontario, Canada A ninferiority Margin for Acne Lesion Counts When comparing different active treatments, a noninferiority also called one-sided equivalence study design is used. This study design requires the definition of a noninferiority margin, the threshold value of clinical relevance. At present, a noninferiority margin of 10 percentage points is conventionally used for the change in acne lesion counts, but it lacks empirical validation. We analyzed the data of 4,081 patients with moderate to severe facial acne. The treatment effect was recorded by the investigator as the relative change in lesion counts from baseline (objective assessment). At the end of the treatment period, patients rated themselves as having their acne condition improved, unchanged, or worsened (subjective assessment). We compared the changes in lesion counts with the patients selfassessment to derive an empirically validated noninferiority margin. We found that an empirically validated noninferiority margin of percentage points for facial acne lesion counts is appropriate. Key Words ninferiority margin; Acne; Choice of delta; Active control Correspondence Address Dr. Christoph Gerlinger, Bayer Schering Pharma AG, Müllerstraße 178, Berlin, Germany ( Christoph.Gerlinger@ bayerhealthcare.com). INTRODUCTION Acne is one of the most common skin diseases (1). The majority of the patients are young individuals in adolescence or early adulthood. Acne is characterized by two major types of lesions: noninflammatory and inflammatory. The count of acne lesions is considered an objective measure for the effect of acne treatment in clinical studies, for example, by the FDA (2). When comparing the efficacy of different active treatments in the reduction of acne lesions, a noninferiority also called one-sided equivalence study design is often used. The methodological principles for the noninferiority trial design are described in the ICH guideline E10 (3). The design of a noninferiority study requires the definition of a noninferiority margin, the threshold value of clinical relevance, in the clinical study protocol. There are statistical and clinical aspects to be considered when choosing a noninferiority margin (4). The major clinical requirement for choosing the noninferiority margin is that any treatment difference smaller than the noninferiority margin should not be of clinical relevance. The major statistical requirement for choosing a noninferiority margin is that the noninferiority margin is small enough to exclude the effect of placebo. The focus of this article is to empirically define the threshold value of clinical relevance for acne lesion counts. It should be noted that the definition of the clinically relevant threshold is independent of the difference between a given treatment and placebo. However, exclusion of the placebo effect needs to be considered when applying the threshold value of clinical relevance in a clinical trial. The data basis for the empirical considerations in this article is a meta-analysis of clinical studies. Four oral contraceptives and one topical gel were assessed in these studies. In all studies, inflammatory and noninflammatory lesions were counted and the treatment effect was assessed by the patient at the end of the studies. The subjective assessment of the treatment effect was used as an anchor for the definition of the noninferiority margin for the acne lesion count. MATERIAL AND METHODS TRIALS INCLUDED Randomized phase 2 and phase 3 clinical trials from four clinical development projects in the acne indication sponsored by Schering AG (today named Bayer Schering Pharma AG) and its subsidiaries between 1994 and 2003 were con- Drug Information Journal, Vol. 42, pp , /2008 Printed in the USA. All rights reserved. Copyright 2008 Drug Information Association, Inc. Submitted for Publication: May 8, 2006 Accepted for Publication: April 25, 2008

2 608 STATISTICS Gerlinger et al. sidered for evaluation. In total, we included seven studies, in which facial acne lesion counts before and after treatment were recorded. For these counts, inflammatory and noninflammatory lesions were distinguished by the investigators (objective assessment). Additionally, a subjective rating of the severity of acne by the volunteers at the end of the trial was recorded, using the categories excellent improvement, good improvement, moderate improvement, or no improvement as well as aggravation (subjective assessment). We excluded studies primarily assessing rosacea as the primary endpoint, since this is considered a different disease entity. Four of our seven trials, carried out by Schering AG in Europe, assessed the effects of oral contraceptives on facial acne. These oral contraceptives contained four different synthetic progesterones in combination with ethinylestradiol at various doses: Monophasic 3 mg drospirenone (DRSP)/30 μg ethinylestradiol (EE) for 21 days (Yasmin) Monophasic 2 mg dienogest (DNG)/30 μg EE for 21 days (Valette) Monophasic 2 mg cyproterone acetate (CPA)/35 μg EE for 21 days (Diane 35) Experimental treatment/35 μg EE for 21 days Triphasic norgestimate (NGM) (180/215/250 μg)/35 μg EE for 7/7/7 days (Pramino) Three further trials were conducted in Europe by Schering AG and in the United States by Berlex Inc., a US subsidiary of Schering AG. These studies compared azelaic acid gel (Finacea) to its vehicle or to azelaic acid 20% cream (Skinoren) with respect to facial acne. The inclusion and exclusion criteria concerning facial acne were comparable in all seven trials. Table 1 lists the seven trials included in this meta-analysis. STATISTICAL METHODS Following the intent-to-treat approach, all randomized patients that provided data were included in the analyses regardless of possible protocol deviations. The treatment effect was recorded by the investigator as the relative change in percentage of inflammatory, noninflammatory, and total lesion counts from baseline. At the end of the treatment period, patients rated whether their acne condition was excellently improved, well improved, moderately improved, unchanged, or worsened. Any missing patient s assessment was replaced by the corresponding dermatologist s assessment because these two ratings showed substantial agreement (weighted κ coefficient 0.64, n = 3,047, 95% CI ) according to the definition of Landis and Koch (5). Missing acne lesion counts were not imputed. In case patients dropped out prematurely, the last available measurement was included in the analysis (last value carried forward method). All variables were analyzed by descriptive statistics, either by absolute and relative frequencies for discrete data, or by the number of nonmissing observations, mean, standard deviation, minimum, first quartile, median, third quartile, and maximum for metric data. Box plots were drawn using 1.5 times the interquartile range (IQR) as the length of the whiskers. Due to the large sample size, outlying observations were not shown in order to improve the readability of the graphs. A nonparametric discriminant analysis with normal kernels and unequal bandwidths (6) was performed separately for each of the relative changes in inflammatory, noninflammatory, and total lesion counts. A parametric discriminant analysis using normal densities was performed additionally as a sensitivity analysis. Sensitivity and specificity for relative changes in total lesion count from baseline were calculated after dichotomization of the patients selfassessments. The patients self-assessments were rated as either improved (patient assessment at end of study: excellent improvement, good improvement, moderate improvement) or not improved (no improvement, aggravation). The corresponding receiver operations characteristics curve (ROC curve) was plotted. All statistical analyses were performed using version 9.1 of SAS software running under Windows XP Professional (7).

3 A ninferiority Margin for Acne Lesion Counts STATISTICS 609 Trials Included in the Analysis Product Treatments Administered Treatment Duration Number of Patients Reference Oral contraceptive 2 mg DRSP/30 μg EE 36 weeks 117 (9,10) 2 mg CPA/35 μg EE 2 mg DNG/30 μg EE 24 weeks 984 (14) NGM (180/215/250 μg)/35 μg EE 2 mg DRSP/30 μg EE 24 weeks 1,116 (15) NGM (180/215/250µg)/35 μg EE 2 mg CPA/35 μg EE 24 weeks 998 (16) experimental treatment NGM (180/215/250 μg)/35 μg EE Topical gel Azelaic acid 15% gel 12 weeks 418 (17) Vehicle gel Azelaic acid 15% gel 12 weeks 416 (18) Vehicle gel Azelaic acid 15% gel 8 weeks 25 (19) Azelaic acid 20% cream TABLE 1 RESULTS A total of 4,081 patients from seven clinical studies were included in the analyses. Of these patients, 3,710 (90.9%) were female because the oral contraceptive trials included only women. Gender distribution was almost equal for the two gel studies, with 56.8% being female. Most patients were Caucasian (93.6%). The mean age was 22.8 years (Table 2). There was no relevant difference in the age distribution for the oral contraceptive and gel studies (data not shown). Of the 4,081 patients, 23.2% subjectively showed excellent improvement, 44.4% good improvement, and 22.8% moderate improvement. A total of 7.8% showed no improvement and 1.8% suffered from subjective aggravation (Table 3). The objective lesion count showed a median reduction of 62.3% for all lesions and a reduction of 72.4% for inflammatory lesions (Table 4). The relation between the objective percentage change in acne lesions and the patients subjective assessments is shown in Figures 1 through 3. There are some discrepancies between the observed changes in the lesion counts and the patients perception of how their acne condition changed; that is, the majority of patients rating themselves as unchanged showed some improvement in acne lesion counts. The nonparametric and parametric discriminant analyses showed discrepant results, especially for the cutoff values between no improvement and aggravation (see Table 5 and Table 6, respectively). This was caused by the fact that the assumption of an approximately normal distribution does not hold for acne lesion counts Drug Information Journal

4 610 STATISTICS Gerlinger et al. TABLE 2 Demographics n (%) Total number of patients 4,081 (100.0) Female 3,710 (90.9) Female, gel studies only (n = 859) 3,473 (56.8) Ethnic group Caucasian 3,819 (93.6) Hispanic 3,124 (3.0) Black 3, 90 (2.2) Age (years; mean 22.81; SD 6.15) Less than 18 3,739 (18.1) 18 to 35 3,137 (76.9) Above 35 3,205 (5.0) TABLE 3 despite our rather high sample size. The data are remarkably skewed and most important there are major floor effects. For example, 405 (9.9%) patients had a percentage change in inflammatory lesions of 100% because they had no residual inflammatory lesions at the end of the study. Estimating normal densities for such data leads to rather flat density estimates that put considerable probability mass (up to 60% in our data set) outside the range of the data. Therefore, we focus on the results of the nonparametric discriminant analysis (Table 5). Subjects with at least 28.20% (cutoff value determined by discriminant analysis) reduction in median total lesion count assessed their facial acne status as at least moderately improved. Subjects with a median total objective lesion n (%) Total number of patients 4,056 (100.0) Subject s assessment Subject s Assessment improvement 1,946 (23.2) improvement 1,810 (44.4) improvement 1,931 (22.8) improvement 1,319 (7.8) Aggravation 1, 75 (1.8) count reduction between 28.20% and an increase of 11.95% regarded their status as not improved, while a median increase of 11.95% or more was perceived as an aggravation. These and the corresponding data for good and excellent improvement are shown in Table 5 and Figure 1. For subjects with inflammatory lesions, an objective reduction of at least 34.71% was regarded as at least a moderate improvement. An objective change of 34.71% to +5.14% was considered as no improvement, and an increase of 5.14% or more was perceived as an aggravation (Table 5). The data for the other subject assessment categories are shown in Table 5 and Figure 2. If the objective lesion counts are used like a diagnostic measurement to distinguish between improved and not improved patients, a margin of 20 percentage points in change from baseline in total lesion count had a sensitivity of 91% and a specificity of 55%. For a margin of 10 percentage points in change from baseline in total lesion count, the sensitivity was 94% and the specificity was 44%. The corresponding receiver operation characteristic curve for the relative change in total lesion count is given in Figure 4. DISCUSSION The aim of this analysis was to derive an empirically validated noninferiority margin for acne

5 A ninferiority Margin for Acne Lesion Counts STATISTICS 611 Acne Lesion Counts Baseline Endpoint Change Percentage Change n = 4,081 Median IQR Median IQR Median IQR Median IQR All lesions ; ; ; ; Inflammatory lesions ; ; ; ; ninflammatory lesions* ; ; ; ; IQR: interquartile range *n = 4,066 for percentage change due to noninflammatory lesion counts of 0 at baseline. TABLE 4 % Change FIGURE 1 Percentage change in total acne lesion counts by subject s assessment Aggravation Subject s Assessment % Change FIGURE 2 Percentage change in inflammatory acne lesion counts by subject s assessment Aggravation Subject s Assessment Drug Information Journal

6 612 STATISTICS Gerlinger et al. FIGURE 3 Percentage change in noninflammatory acne lesion counts by subject s assessment. % Change Aggravation Subject s Assessment lesion counts. The data used for the analysis were derived from seven clinical trials including 4,081 patients with moderate to severe facial acne. For the determination of the noninferiority margin in acne lesion count, we mapped a clinical measure that describes (patho)morphological and (patho)physiological changes in the skin to an anchor that determines the clinical relevance of theses changes. The change in acne lesion count recorded by the physician is regarded as an objective mea- TABLE 5 Cutoff Points for Acne Lesion Counts in Percentage According to nparametric Discriminant Analysis Aggravation All lesions 100 to under to under to under to or more under Inflammatory lesions 100 to under to under to under to under 5.14 or more ninflammatory lesions 100 to under to under to under to under or more TABLE 6 Cutoff Points for Acne Lesion Counts by Percentage According to Parametric Discriminant Analysis Aggravation All lesions 100 to under to under to under to under or more Inflammatory lesions 100 to under to under to under to under or more ninflammatory lesions 100 to under to under to under to under or more

7 A ninferiority Margin for Acne Lesion Counts STATISTICS 613 Sensitivity Specificity sure that best describes the change in the skin condition of the patients with facial acne. For the assessment of clinical relevance, we used the patients subjective ratings of their change in acne condition. Using patients self-assessment of a change in skin condition for determining the clinical relevance of (patho)morphological and (patho)physiological changes was proposed by Schmitt (8). The vast majority of patients perceived an objective reduction of acne lesions subjectively also as an improvement, giving evidence that objective and subjective assessments are concordant. A minority regarded a small objective reduction as no improvement or even as aggravation. A reduction of acne lesions from baseline by 25% is often regarded as no relevant change for the individual level (9 11). However, for the definition of clinically relevant difference, not only the difference between no improvement and any improvement should be considered. The difference between the stages of improvement gives additional information. This is especially true for acne treatments whose aim is complete remission. Descriptive analysis of all categories of patients ratings of their change in acne condition showed a clear relation between these ratings and the change in lesion count. The median values for the percentage changes in total lesion count of the different rating classes are about 20% apart from each other. Comparable findings were seen for the analysis of the subgroups of inflammatory and noninflammatory lesions. These findings were confirmed by the discriminant analyses. The observation that patients described their skin conditions as unchanged although there was a reduction in lesion count compared to baseline supports the view that the noninferiority margin should not be chosen too small to ensure that it is clinically relevant. This is supported by the analysis of sensitivity and specificity. We estimated from the data sensitivity and specificity of a noninferiority margin for classifying patients as improved (patient assessment at end of study: excellent improvement, good improvement, moderate improvement) or not improved (no improvement, aggravation) using the percentage change in total lesion count from baseline. A relative decrease in total lesion count of 20% has a high sensitivity (over 90%) with a reasonable specificity (over 50%). For a relative decrease in total lesion count of 10%, specificity is reduced to 44% with only a slight improvement in sensitivity. The corresponding ROC curve shows that even smaller margins would result in high losses of specificity. From the data in the previous paragraphs, we consider that a difference of 20 percentage points in acne lesion counts between treatments is already clinically relevant and that a difference of below 10 percentage points is not clinically relevant. The determination of the clinically relevant noninferiority margin did not consider the statistical requirement that a noninferiority margin is limited by the effectiveness of the reference treatment studied with respect to placebo. Any noninferiority margin should be smaller than the difference between the reference active treatment and placebo. For example, Lucky et al. (12) and Redmond et al. (13) reported treatment differences of 20 percentage points between triphasic norgestimate/ ethinylestradiol and placebo. Since the treatment difference between active and placebo varies from trial to trial, it is reasonable to choose a margin of 15% in a noninferiority trial FIGURE 4 Receiver operations characteristics curve. Drug Information Journal

8 614 STATISTICS Gerlinger et al. with triphasic norgestimate/ethinylestradiol as reference. With this margin, the statistical requirement of exclusion of placebo response should be fulfilled. This 15% margin is in line with the clinically relevant difference of 10 20% found in our analyses. Therefore, a noninferiority study to compare an active treatment with triphasic norgestimate/ethinylestradiol using a 15% noninferiority margin would fulfill both the major clinical requirement and the major statistical requirement for noninferiority margins. As acne is a non-life-threatening, mostly cosmetic medical problem, the subjective perception of the patient is key. Any objective difference between treatments that is not regarded by the patient as relevant is of no clinical relevance, independent of the objective baseline value or the efficacy of the treatments compared. We need to emphasize that this is true for acne and might be true for other nonserious subjective ailments. However, this cannot be generalized to more serious diseases or diseases in which the patient s perception is irrelevant, such as hypertension. CONCLUSION In summary, a noninferiority study should be able to detect relevant differences between treatments and to determine that there are no relevant differences for equally effective treatments. The data showed that in noninferiority studies in acne, the noninferiority margin should not be smaller than 10 percentage points and not larger than 20 percentage points in change in acne lesion from baseline counts based on clinical considerations. However, before designing a noninferiority study, not only the clinical aspect of relevance should be considered but also the effect of the reference treatment. This might lead to a smaller noninferiority margin, as seen in our example, where statistical requirements led to a noninferiority margin that should not exceed 15 percentage points. Acknowledgments The authors thank the anonymous reviewers, whose comments substantially improved our article. REFERENCES 1. James WD. Clinical practice: acne. N Engl J Med. 2005;352: US Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research. Clinical/medical guidance for industry acne vulgaris: developing drugs for treatment. September Available at: accessed January 16, ICH (2000). International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH harmonised tripartite guideline. Choice of control group and related issues in clinical trials. Recommended for adoption at step 4 of the ICH process on July 20, 2000, by the ICH steering committee. 4. Committee for Medicinal Products for Human Use. Guideline on the Choice of the n-inferiority Margin. London, July 27, Doc. Ref. EMEA/CPMP/EWP/2158/99. Available at: en.pdf, accessed October 31, Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1): SAS Institute Inc SAS/STAT 9.1 User s Guide. Cary, NC: SAS Institute Inc. 7. SAS Institute Inc. Available at: 8. Schmitt GJ. Scale development for clinical assessment. In: Schwindt AD, Maibaich HI, eds. Cutaneous Biometrics. New York: Kluwer; Loock W. Double-blind, randomized, multicenter study with SH T 470 FA in comparison with Diane -35 in women with acne over 9 treatment cycles. Schering AG internal research report number AM80, June 18, van Vloten WA, van Haselen CW, van Zuuren EJ, Gerlinger C, Heithecker R. The effect of 2 combined oral contraceptives containing either drospirenone or cyproterone acetate on acne and seborrhea. Cutis. 2002;69(4 Suppl): Alosh M, Wilkin J. Assessing the relationship between investigator global evaluation and acne lesion counts. Drug Inf J. 2004;38: Lucky AW, Henderson TA, Olson WH, et al. Effectiveness of norgestimate and ethinyl estradiol in treating moderate acne vulgaris. J Am Acad Dermatol. 1997;37:

9 A ninferiority Margin for Acne Lesion Counts STATISTICS Redmond GP, Olson WH, Lippman JS, et al. rgestimate and ethinyl estradiol in the treatment of acne vulgaris: a randomized, placebo-controlled trial. Obstet Gynecol. 1997;89: Bongartz M. Multicenter, double-blind, randomized parallel group study on efficacy of 0.03 mg ethinylestradiol and 2 mg dienogest in comparison to triphasic ethinylestradiol and norgestimate over six cycles in patients with acne papulopustulosa. Schering AG internal clinical study report number A07062, June 21, Wack C. Multicenter, double-blind, randomized parallel group study on efficacy of 0.03 mg ethinylestradiol and 3 mg drospirenone (DRSP) in comparison to triphasic ethinylestradiol and norgestimate (NGM) over six cycles in patients with mild to moderate acne papulopustulosa. Schering AG internal clinical study report number A07148, February 25, 2003, as amended. 16. Schneider A. Multicenter, double-blind, randomized parallel group study on efficacy of mg ethinylestradiol/2 mg cyproterone acetate and of mg ethinylestradiol/2 mg cyproterone acetate in combination with 10 mg cyproterone acetate in comparison to triphasic ethinylestradiol/norgestimate over 6 cycles in women with acne papulopustulosa. Schering AG internal clinical study report number A18566, July 2, Zimmermann D. A 12-week, randomized, doubleblind multicenter study comparing the clinical efficacy and safety of azelaic acid 15% gel (SH H 655 BA) with its vehicle (SH H 655 PBA) in patients with mild to moderate acne. Berlex Inc. internal clinical study report number A09241, September 16, A 12-week, randomized, double-blind multicenter study comparing the clinical efficacy and safety of azelaic acid 15% gel (SH H 655 BA) with its vehicle (SH H 655 PBA) in patients with mild to moderate acne. Berlex Inc. internal clinical study report number A09242, September 16, Gaupe K. An 8-week controlled, double-blind pilot study comparing the initial clinical effect of 15% azelaic acid hydrogel SH H 655 BA with 20% azelaic acid cream (Skinoren ) during twice-daily topical treatment of patients with papulopustular facial acne. Schering AG internal clinical study report number AU36, January 15, The authors report no relationships to disclose. Drug Information Journal