A simulation using data from a primary care practice database closely replicated the women s health initiative trial

Transcription

1 Journal of Clinical Epidemiology 60 (2007) 686e695 A simulation using data from a primary care practice database closely replicated the women s health initiative trial Richard L. Tannen a,b, *, Mark G. Weiner a, Dawei Xie b, Kurt Barnhart b,c a Department of Medicine, University of Pennsylvania School of Medicine, 295 John Morgan Building, 36th & Hamilton Walk, Philadelphia, PA 19104, USA b Center for Clinical Epidemiology and Biostatistics, University of Pennsylvania School of Medicine, 295 John Morgan Building, 36th & Hamilton Walk, Philadelphia, PA 19104, USA c Department of Obstetrics & Gynecology, University of Pennsylvania School of Medicine, 295 John Morgan Building, 36th & Hamilton Walk, Philadelphia, PA 19104, USA Accepted 9 October 2006 Abstract Objective: In contrast to prior observational studies, hormone replacement therapy (HRT) did not prevent coronary heart disease in the Women s Health Initiative Randomized Controlled Trial (WHI RCT). To assess the validity of a novel observational study design, we compared the WHI RCT with a simulation using data from the United Kingdom General Practice Research Database (GPRD). Study Design and Setting: A cohort from GPRD was used to simulate the WHI RCT by replicating, to the extent possible, all aspects of the RCT except randomization. The study included 37,730 Unexposed and 13,658 Exposed women treated with estrogen and norgestrel. Results: Myocardial infarction (adjusted hazard ratio 0.95 [0.78e1.16]) was not decreased significantly in the GPRD Exposed group. Similar to the WHI RCT, stroke, venous thromboembolic events, and breast cancer were increased; and colorectal cancer was decreased. Although death appeared to decrease in the total cohort, it was unaltered in a subset of subjects without missing data on baseline covariates. Conclusion: A structured comparison using data from GPRD was largely concordant with the WHI RCT and did not show a cardioprotective effect of HRT. These findings further generalize the results of WHI and reinforce the potential utility of this analytic approach Ó 2007 Elsevier Inc. All rights reserved. Keywords: Cohort studies; Coronary heart disease; Databases; Hormone replacement therapy; Menopause; Randomized controlled trial 1. Introduction Considerable controversy exists concerning the role for observational studies in informing the practice of medicine [1e10]. Several publications report similar results from observational studies and randomized controlled trials (RCTs) and conclude they are useful for evaluation of therapeutic effectiveness [1e3,11e13]. However, most of these comparisons were not between similar study designs except for randomization [2,3,14,15]. Others are unconvinced about the value of observational studies arguing that lack of randomization subjects a study to unidentified bias [5,6]. A frequently cited example is the observational studies finding that hormone replacement therapy (HRT) protects postmenopausal women from * Corresponding author. Tel.: ; fax: address: tannen@mail.med.upenn.edu (R.L. Tannen). development of coronary heart disease [10,16e20]; whereas the Women s Health Initiative (WHI) and the Heart and Estrogen/Progestin Replacement Study (HERS) RCTs did not confirm this result [21e24]. To assess the validity of a novel observational study design, we initiated studies using a primary care practice database, the United Kingdom General Practice Research Database (GPRD), as the source of observational data. To the extent possible, with the exception of treatment randomization, previously performed RCTs are simulated using data from the GPRD. The feasibility of this strategy was demonstrated by a study of systolic hypertension in the elderly, which replicated the important findings in the RCT [25]. To further validate this methodology and to study an important controversial topic, we investigated the WHI RCT [21]. Our findings are informative in regard to the validity of our approach to observational studies as well as to the use of HRT in postmenopausal women /07/$ e see front matter Ó 2007 Elsevier Inc. All rights reserved. doi: /j.jclinepi

2 R.L. Tannen et al. / Journal of Clinical Epidemiology 60 (2007) 686e Methods 2.1. United Kingdom general practice research database The UK General Practice Research Database (GPRD), described in detail previously, has compiled information from the electronic medical records of Primary Care Practices encompassing a representative sample of approximately 5.7% of the UK population over the time frame from 1990e2000 and contains records of over eight million patients [25,26]. It includes the complete primary care medical record, detailed information on essentially all medications prescribed, as well as information received from outside consultants and hospitalizations. Clinical data on the patients is compiled centrally with personal identifiers removed, thus preserving anonymity. This project has been approved by the GPRD Scientific and Ethical Advisory Group and by the University of Pennsylvania Institutional Review Board for Research Involving Human Beings WHI RCT and GPRD study protocols The GPRD study was designed to simulate the WHI RCT except for randomization. Key features of the WHI RCT are summarized in Table 1 and compared with the GPRD protocol. Because the initial version of the GPRD database had a marked diminution in patients after the year 2000, a similar study duration and subject recruitment interval of the first 5 years was used, but the GPRD study began and ended 3 years earlier (1/1/90e4/30/1999) than the WHI RCT. The primary outcome in the WHI RCT was coronary heart disease (CHD), a combination of nonfatal myocardial infarction and death attributable to coronary heart disease. Because CHD death could not be assessed accurately, the GPRD study used total myocardial infarction. Breast cancer, endometrial hyperplasia or cancer, pulmonary embolus, deep venous thrombosis, and malignant melanoma were study stop points. Other outcomes were stroke, osteoporotic fractures, and cancer. Newly occurring angina in subjects without a myocardial infarction also was analyzed, but Table 1 Comparison of WHI RCT and GPRD observational study Study protocol WHI RCT GPRD Study dates and enrollment Study dates 1993eApril eApril 1999 Enrollment Until 1998 Until 1995 Participant age 50e79 55e79 Menopausal status 1 year Assumed menopausal based on age amenorrhea (age 50e54) and O6 months (age O 55) Any use of post menopausal hormone therapy Exclusion criteria Hysterectomy Acute MI, CVA, or TIA within 6 months entry H/O breast or endometrial cancer H/O malignant melanoma H/O other malignancies in past 10 years a Abnormal PAP smear, pelvic exam Endometrial hyperplasia H/O nontraumatic pulmonary embolus or DVT Severe hypertension Chronic hepatitis or cirrhosis Corticosteroid, tamoxifen, or anticoagulant Rx at entry Medical condition b with predicted survival!3 years Condition inconsistent with study adherence c Exposure Conjugated Estrogen mg, PO, daily Medroxyprogesterone 5.0 mg, PO, daily Norgestrel 150 mg, PO, days 17e28 Abbreviations: MI, myocardial infarction; CVA, cerebrovascular accident; TIA, transient ischemic attack; DVT, deep venous thrombosis. a Exclusive of nonmelanotic skin cancers. b Conditions included in GPRD study were AIDS, CPD, heart failure, renal impairment. c Conditions included in GPRD study were alcoholism, drug dependency, dementia, mental illness.

3 688 R.L. Tannen et al. / Journal of Clinical Epidemiology 60 (2007) 686e695 not compared with the WHI RCT, which reported angina requiring hospitalization Entry criteria, treatment protocol, and start time Fig. 1 outlines the selection process for subjects included in the GPRD study. All females aged 50e79 and then those treated with any estrogen containing preparation during the recruitment interval were identified. An insufficient number received conjugated estrogen (0.625 mg daily) and mexdroxyprogresterone, the HRT used in the WHI. Therefore women taking combination conjugated estrogen (0.625 mg daily) and norgestrel (150 mg on days 17e28 of the cycle) were considered potential Exposed subjects. Those taking other HRT preparations were eliminated, and women not receiving any HRT preparation during the recruitment interval were potential Unexposed subjects. Next, start time was defined and exclusion criteria applied. Exposed group start time was the first date during the recruitment interval when a woman both met the age criteria and first took the HRT preparation. The Exposed group was finalized by eliminating women with study exclusion criteria. Potential Unexposed subjects were age matched to this Exposed group using a computer generated random number selection program with a goal of three matches for each Exposed subject. Unexposed women s start time was considered identical to their matched Exposed subject, and this group was finalized by eliminating women with exclusion criteria. WHI RCT used a predetermined percentage of patients in each age category between 50 and 79 years and was heavily weighted to the older age group, very few of who receive HRT in the practice setting. To achieve a mean age more similar to the WHI study, the initial GPRD study was restricted to women aged 55e79. A second study examined women aged between 50 and 55 years [27]. FEMALE Age Recruitment Interval (n = 905,234) Excluded use non study HRT drug (n = 71,150) Excluded study exclusion criteria (n = 4369) HRT use Recruitment Interval (n = 109,831) Study HRT Drug (n = 38,681) Exposed Cohort Ages (n = 34,312) No HRT use Recruitment Interval (n = 795,403) Random Matched to Exposed Cohort (n = 75,757) Unexposed Cohort Ages (n = 67,832) Excluded study exclusion criteria (n =7925) Excluded age (n = 20,654) Excluded age (n = 30,102) Exposed Final Cohort Ages (n = 13,658) Unexposed Final Cohort Ages (n = 37,730) Study stopped by lost f/u event (n = 6879) Study stopped by lost f/u event (n = 19,258) Completed Study (n = 6779) Completed Study (n = 18,472) Fig. 1. Flow diagram for selection of study participants from the GPRD.

4 R.L. Tannen et al. / Journal of Clinical Epidemiology 60 (2007) 686e Exclusion criteria replicated those used in the RCT (Table 1), and outcome stop points were similar. Patients were considered lost to follow-up if they left the practice before a stop point or the end-study date, or if the practice was dropped from GRPD Data analysis Outcomes similar to those in the RCT were analyzed by both: 1. a simulated Intention to Treat analysis in which, regardless of subsequent use of HRT preparations in either the Exposed or Unexposed groups, the study continued until attainment of a stop point, a lost to follow-up event (vide supra), or the study end date. 2. an As-Treated analysis, in which treatment of the Unexposed group with any HRT preparation was considered a stop point on the date when the second prescription of an HRT was ordered. In the Exposed group, the end date of the last HRT prescription s duration (whether the study drug or another estrogen containing preparation) was the stop point. Two subsets of the entire cohort also were analyzed: 1. A Without Missing Data subset was restricted to all subjects not missing any data on the following baseline confounders: systolic blood pressure (SBP), BMI, and smoking. 2. A Without Prior HRT subset included only subjects not exposed to any HRT preparation in the year preceding the study start date Statistical analysis Hazard ratios (HRs) were determined using Cox proportional hazard analysis. Confounders utilized for the adjusted HR included age, blood pressure (BP), Body Mass Index (BMI), smoking, previous myocardial infarction or stroke, and evidence for angina or other ischemic cardiovascular disease, transient ischemic attack or other ischemic cerebrovascular disease, heart failure, or diabetes mellitus. An analysis including hypercholesterolemia as a confounder also was performed and did not differ meaningfully from the analyses without this potential confounder. In view of missing data for BP, BMI, and smoking, multiple imputations for the missing data were carried out based on all potential confounders and outcomes. Five imputed data sets were combined in the analysis of the adjusted hazard ratios [28e30]. Outcomes also were analyzed using propensity scores analysis, which accounts for missing confounders in a different fashion than the multiple imputation method. Outcome hazard ratios were determined for each propensity score quintile, and the five-quintile hazard ratios combined to determine an overall hazard ratio using a Cox model treating the five quintiles as strata. Because the number of confounders used in a propensity score analysis are not limited; all the baseline pertinent demographic information, cardiovascular risk factors, cardiovascular diagnoses, and cardiovascular drugs used were included in this analysis. The propensity score methodology for handling missing values has been described previously [31e33]. Hazard ratio differences between studies were analyzed statistically as described previously, using a standard normal z-test where the z-score was obtained from the difference between the logarithm of the hazard ratios divided by the standard error of that difference [25]. 3. Results 3.1. Baseline characteristics Baseline characteristics and number of the subjects in the WHI RCT and GPRD study are shown in Table 2. Subjects in the GPRD study were younger, less obese, more were current smokers, and had a lower incidence of diabetes, hypercholesterolemia, and hypertension (mean SBP was higher because fewer subjects were receiving antihypertensive therapy). Prior use of HRT was greater in the GPRD Exposed group than the RCT Treated group, and lower in the GPRD Unexposed group than the RCT Placebo group. In the RCT, baseline characteristics of the Placebo and HRT arms were virtually identical (Table 2). In the GPRD study, the Exposed group had substantially reduced risk factors for cardiovascular disease in comparison to the Unexposed group. In addition, HRT treatment before study entry was much more prevalent in the Exposed group. Data was missing for at least one parameter (SBP, BMI, or smoking) in 21.1% of the Exposed and 35.9% of the Unexposed groups Outcomes Simulated intention to treat analysis Outcomes for the RCT and the GPRD study are shown in Table 3 and Fig. 2. Average follow-up time was similar in both arms of the GPRD Study and comparable to the RCT Unexposed GPRD vs. Placebo RCT comparisons Event rates for myocardial infarction, breast cancer, and stroke did not differ significantly between the GPRD Unexposed and RCT Placebo Groups (Table 3). Rates for death, deep vein thrombosis (DVT), and venous thromboembolism (VTE) were higher in the GPRD Unexposed group (GPRD codes include some nondvt, which may account for the disparity in DVT and VTE). Other rates were lower in the GPRD Unexposed group.

5 690 R.L. Tannen et al. / Journal of Clinical Epidemiology 60 (2007) 686e695 Table 2 Baseline characteristics a WHI RCT GPRD study HRT (n ) Placebo (n ) Exposed (n 5 13,658) Unexposed (n 5 37,730) P (Exp vs. Unexp) Age (years) NS 50e59 (%) ! e69 (%) ! e79 (%) !0.01 BMI (kg/m 2 ) Mean !0.01 O30 (%) !0.01 SBP (mmhg) !0.01 Hypertension (%) !0.01 Smoker Past (%) NS Current (%) !0.01 Diabetes (%) !0.01 High Chol (%) !0.01 Prior MI (%) b !0.01 Prior CVA (%) b !0.01 Fracture (%) c !0.01 Angina (%) NS Prior HRT use Past (%) !0.01 Current (%) d !0.01 Abbreviations: Chol, cholesterol; MI, myocardial infarction; CVA, cerebrovascular accident. a Some subjects in the GPRD study were missing data on BMI, SBP, or smoking. The data shown for these three parameters exclude these subjects. b Prior MI or CVA refers to events more than 6 months before start of study. c Fracture refers to fractures occurring at age 55 or older. d Current HRT use was within 3 months of study start in the WHI RCT and 6 months in the GPRD study Exposed (HRT) vs. Unexposed (Placebo) comparison COX unadjusted and adjusted hazard ratios are shown for the GPRD study in Table 3. GPRD adjusted hazard ratios were increased significantly for DVT, VTE, breast cancer, and all cancer; and reduced for colorectal cancer and death. These results are similar to the WHI RCT with the exception of all Cancer and death, which were unchanged in the RCT. The GPRD also differs from the RCT in regard to myocardial infarction, which increased in the RCT and was unchanged in the GPRD study. Angina was increased in the GPRD study (Adj HR 1.33 [95% CI:1.18, 1.51]); whereas angina requiring hospitalization was unchanged in the RCT. Endometrial cancer was not changed significantly in the GPRD study (Adj HR 1.39 [95% CI: 0.87,2.20]), and did not differ Table 3 GPRD intention to treat analysis WHI RCT GPRDeintention to treat HRT a Placebo a HR 95% CI Exp a Unexp a HR 95% CI Adj HR 95% CI P b F/U Time (year) MI e e e Stroke e e e DVT e e e PE e e e VTE e e e Breast CA e e e Colorectal CA e e e Cancer e e e Hip Fracture e e e Death e e e Abbreviations: MI, myocardial infarction; DVT, deep venous thrombosis; PE, pulmonary embolus; VTE, venous thromboembolic event; CA, cancer. a Events per 100 patient years. b P reflects analysis of RCT HR vs. GPRD adjusted HR.

6 R.L. Tannen et al. / Journal of Clinical Epidemiology 60 (2007) 686e MI Stroke WHI RCT GPRD ITT total cohort GPRD ITT no missing data GPRD ITT no prior HRT GPRD As Treated total cohort GPRD As Treated no prior HRT VTE Breast Cancer Colorectal Cancer Hip Fx Death Fig. 2. Comparison of hazard ratios. The hazard ratios (HRs) and confidence limits for the WHI RCT are taken from the WHI publication [21]. For comparison, the adjusted HRs for both the intention to treat analyses and the as treated analyses of the GPRD study, are shown. Abbreviations: MI, myocardial infarction; VTE, venous thromboembolic event; Hip Fx, hip fracture; ITT, simulated intention to treat analysis. significantly from the RCT results HR 0.83 (95% CI: 0.47,1.47). Most hazard ratios in the GPRD study were similar in direction to those in the RCT. However, comparisons between the GPRD adjusted and the RCT hazard ratios were significantly different for myocardial infarction, death, pulmonary embolus, and breast cancer (Table 3) Drug modifier No substantive differences were detected between the Exposed and Unexposed groups in use during the study of medication that could modify cardiovascular events including: anticoagulant and antiplatelet drugs, antihypertensive drugs, cholesterol lowering agents, nitrates, and digitalis Statistical adjustments Given the importance of the differences between the unadjusted and adjusted hazard ratio for myocardial infarction and stroke in the GPRD study, and the large percentage of missing data on SBP, BMI, and smoking, other analytic approaches were used to verify the Cox adjusted analysis, which used multiple imputations for missing data. Propensity score analysis, which treats missing data in a different fashion, resulted in adjusted hazard ratios similar to those with multiple imputations (Table 4). Also subjects without any missing data for SBP, BMI, or smoking exhibited results largely similar to the overall GPRD cohort (Table 5 and Fig. 2). Of note, death was not decreased significantly in the without missing data subset with an adjusted hazard ratio of 0.91 (95% CI: 0.79e1.05) Subset without Prior HRT ingestion The adjusted hazard ratios for the subset that did not take any HRT preparation during the 1 year preceding the study start date were substantively the same as the total GPRD cohort (Table 5 and Fig. 2) As treated analysis The results of the as treated analysis are shown in Table 6 and Fig. 2. Average follow-up duration was markedly lower in the Exposed group of the as treated analysis (3.64 years) than the Unexposed group (5.01 years). Adjusted hazard ratios (including angina) were largely similar in the as treated and intention to treat analyses with the exception of death, which was significantly lower in the as treated analysis. Table 4 Comparison of adjusted hazard ratios (HR) from multiple imputations and propensity score analyses GPRD intention to treat Multiple imputations Propensity score HR 95% CI HR 95% CI MI e e1.17 Stroke e e1.42 DVT e e1.73 PE e e1.83 VTE e e1.69 Breast CA e e1.91 Colorectal CA e e0.89 Cancer e e1.32 FX Hip e e1.24 Death e e0.89

7 Table 5 GPRD intention to treat dcomparison of total cohort and subsets Total cohort No missing data subset No prior HRT subset Adj HR 95% CI Exp a (n 5 10,778) Unexp a Exp a (n 5 24,175) HR 95% CI Adj HR 95% CI P b P c (n 5 6,924) Unexp a (n 5 37,568) HR 95% CI Adj HR 95% CI P b P c F/U Time (year) MI e e e e e Stroke e e e e e DVT e e e e e PE e e e e e VTE e e e e e Breast CA e e e e e Colorectal e e e e e CA Cancer e e e e e Hip fracture e e e e e Death e e e e e a Events per 100 patient years. b P Adjusted HR vs. GPRD total cohort. c P Adjusted HR vs. RCT. Table 6 GPRD as treated analysis Total cohort No prior HRT subset Exp a Unexp a HR 95% CI Adj HR 95% CI P b P c (n 5 6,924) Exp a Unexp a (n 5 37,568) HR 95% CI Adj HR 95% CI P d P c F/U Time (year) MI e e e e Stroke e e e e DVT e e e e PE e e e e VTE e e e e Breast CA e e e e Colorectal CA e e e e Cancer e e e e Hip fracture e e e e Death e e ! e e a Events per 100 patient years. b P Adjusted HR vs. GPRD intention to treat total cohort. c P Adjusted HR vs. RCT. d P Adjusted HR vs. GPRD as treated total cohort. 692 R.L. Tannen et al. / Journal of Clinical Epidemiology 60 (2007) 686e695

8 R.L. Tannen et al. / Journal of Clinical Epidemiology 60 (2007) 686e Comparisons to the total as treated cohort between the subset without prior HRT (Table 6 and Fig. 2) and the no missing data subset (not shown) were similar to the comparisons between these subsets and the total cohort in the intention to treat analyses. 4. Discussion The implications of this study are twofold: (1) they are relevant to the controversy concerning the use of HRT in postmenopausal women and (2) they provide evidence supporting the validity of observational studies using data from a primary care electronic medical record database. In contrast to several prior observational studies, we did not find a protective effect of HRT on the occurrence of myocardial infarction [18e20,34,35]. This finding, detected with either a simulated intention to treat or an as treated analysis, is consistent with the RCT s, WHI, and HERS, which did not find a protective effect of HRT [21e24]. However, the WHI RCT found that HRT had an adverse effect on the occurrence of myocardial infarction [21,22]. We did not replicate this finding, which might be accounted for by differences in progestin type, dose, or cyclic administration. However, we did find an increase in angina. Of interest, a recent GPRD study simulating the WHI RCT in a fashion somewhat similar to our study also did not find a reduction in coronary heart disease with HRT [36]. Our findings are important because they reflect what happens to a nonselected, reasonably large sample of the overall population. In that regard, they differ from prior observational studies and also from the subject selection constraints of any RCT. Perhaps most important, they generalize the results of the WHI RCT to all women who meet the entry criteria for this study. Our results also confirm that women in the general population who receive HRT have a lower risk profile for the development of cardiovascular disease than those who are untreated [34,37]. In view of universal health care in the UK, access to physicians or the cost of drugs should not account for this disparity. The GPRD study preceded publication of the recent RCT s [21e24], so they did not influence the choice of HRT. Presumably it resulted from physician reluctance to prescribe HRT to women with cardiovascular risk factors [10,37]. In addition to lack of protection against coronary artery disease, our results were similar to the WHI RCT for almost every other major outcome. HRT therapy increased the likelihood of stroke, breast cancer, and venous thrombotic events and reduced the development of colorectal cancer; consistent with the results of both prior RCT s and observational studies [34,35]. The most perplexing difference between the GPRD study and WHI was in the death outcome; which was unaltered in the RCT and reduced significantly in the Exposed group of the GPRD study. Several analyses suggest this may be spurious and related to more missing data in baseline covariates in the Unexposed group. In the subset without missing data the only outcome that differed from the total cohort was death, which exhibited an adjusted hazard ratio similar to the WHI RCT. Furthermore, in quintile five of the propensity score analysis with less than 4% missing data, the death hazard ratio was 1.0, whereas the other quintiles with more missing data had reduced death hazard ratios (0.64e0.81). What accounts for the apparent aberrant death outcome in combination with missing data is unresolved; however, it suggests caution in the interpretation of GPRD studies with a large disparity in the amount of key missing confounding data between the Exposed and Unexposed groups. Overall, despite the potential problems enumerated, the reasonable similarity between the results of the GPRD observational study and the WHI RCT supports the view that observational studies can provide valid information concerning therapeutic effectiveness. In this study despite a striking difference in the baseline characteristics as well as a disparity in missing data between the Exposed and Unexposed groups, both Cox multiple imputations and the propensity score adjusted hazard ratios largely replicated the RCT. Our results were not adjusted for socioeconomic factors, which may influence differences in myocardial infarction between observational studies and RCT s of HRT [33,38]; however, HRT use in the UK appears to vary only slightly based on socioeconomic factors [37]. Most important, in contrast to many observational studies of HRT that showed a reduction in coronary artery disease, this study demonstrates that a large database of patient care medical records that captures a true sample of the overall population can identify and/or overcome selection bias and with appropriate statistical adjustment provide valid results [16e20,34,35]. A prior study, which compared the Syst Eur RCT with a simulated RCT using the GPRD database also showed excellent concordance between the observational study and RCT [25]. However, the baseline characteristics and the amount of missing data were similar in the Exposed and Unexposed groups, and the results required essentially no statistical adjustments. The current GPRD study addressed a potentially much more difficult analytic problem, but still performed admirably. Several limitations to interpretation of this study, in addition to those already addressed, bear consideration. The HRT preparation in the GPRD study differed from the WHI study. Estrogen treatment was identical, but progestin therapy differed with norgestrel taken on the last 11 days of the cycle rather than daily medroxyprogesterone [21]. Medroxyprogestorene acetate is derived from progesterone whereas norgestrel is a 19-nortestosterone derivative. Whether progestin s used continuously or with different androgenicity influence cardiovascular or breast cancer risk is unresolved [39e44]. The WHI RCT required convincing evidence for menopause, which could not be ascertained in the GPRD study.

9 694 R.L. Tannen et al. / Journal of Clinical Epidemiology 60 (2007) 686e695 However, women in the GPRD study were aged 55 or older, so 95% would be expected to be postmenopausal [45]. Mean age was higher in the WHI RCT and 21.7% of subjects were aged 70e79 years, as contrasted with 2.7% in the GPRD study. This difference resulted from enrollment of predefined age groups in the WHI study, whereas in practice treatment with HRT is rare in women over the age of 70 years. However, despite these differences between the RCT and the GPRD study, the results were largely similar. Finally, HRT use before study entry, a potential confounding problem, differed between the WHI and GPRD studies [46]. Approximately 26% of women had taken HRT before entry into WHI, whereas in the GPRD study 54% of the Exposed and only 1.9% of the Unexposed group received prior HRT. Thus, potentially prevalent user bias might exist, a possibility reinforced by the publication of the observational study arm of the WHI [46,47]. Results from the GPRD subset that had not taken any HRT during the year before study entry, analyzed using either a simulated intention to treat or an as treated analysis were largely similar to the overall GPRD cohort, suggesting no important influence of prior HRT use on our results. Features of the GPRD study that distinguish it from prior observational studies and might account for the difference in results include the following. Subject selection is an unbiased sample because it does not represent volunteers but rather is a representative sample of the entire UK population. Abundant information in the database on the characteristics of the subjects provides a greater opportunity to detect bias between the Exposed and Unexposed groups. Additionally, the nature of the NHS, where everyone is eligible for both health care and medications without cost, might mitigate against bias. Finally, in contrast to prior observational studies of HRT, the GPRD study had similar entry criteria and a treatment protocol comparable to the WHI RCT, so the only major difference was absence of randomization. As suggested previously, this more structured comparison between the results of RCTs and observational studies should provide the most rigorous evidence for actual concordance, or lack thereof, between these two study designs. Indeed, the recently published observational study arm of the WHI and GPRD simulation of the WHI also were more concordant with the RCT than most prior observational studies [36,47]. This is in contrast with two other studies, which used the GPRD using other methodology and found an HRT associated reduction in myocardial infarction [48,49]. Additional GPRD simulation studies are required to determine whether specific study characteristics correlate with the likelihood of observational study validity. Acknowledgments This study was supported by grant RO1-HL from the National Institutes of Health. The authors gratefully acknowledge the helpful advice of Dr. Jesse Berlin in planning of the study, and Drs. James Lewis, Stephen Kimmel, and Brian Strom for their critical review of the manuscript. Dr. Sue Marcus provided advice for the propensity score analysis, and Ms. Xingmei Wang assisted with the biostatistical analyses. References [1] Ioannidis JPA, Haidich AB, Pappa M, Pantazis N, Kokori SI, Tektonidou MG, et al. Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA 2001;286:821e30. [2] Britton A, McKee M, Black N, McPherson K, Sanderson C, Bain C. Choosing between randomised and non-randomised studies: a systematic review. Health Technol Assess 1998;2(13). [3] MacLehose RR, Reeves BC, Harvey IM, Sheldon TA, Russell IT, Black AMS. A systematic review of comparisons of effect-sizes derived from randomized and non-randomised studies. Health Technol Assess 2000;4(34). [4] Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ 1996;312:1215e8. [5] Kunz R, Oxman AD. The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials. BMJ 1998;317:1185e90. [6] Popcock SJ, Elbourne DR. Randomized trials or observational tribulations? N Engl J Med 2000;342:1907e9. [7] McMahon AD. Observation and experiment with the efficacy of drugs: a warning example from a cohort of nonsteroidal antiinflammatory and ulcer-healing drug users. Am J Epidemiol 2001;154:557e62. [8] MacMahon S, Collins R. Reliable assessment of the effects of treatment on mortality and major morbidity, II: observational studies. Lancet 2001;357:455e62. [9] Califf RM, Pryor DB, Greenfield J Jr. Beyond randomized clinical trials: applying clinical experience in the treatment of patients with coronary artery disease. Circulation 1986;74:1191e4. [10] Vandenbroucke JP. When are observational studies as credible as randomised trials? Lancet 2004;363:1728e31. [11] McKee M, Britton A, Black N, McPherson K, Sanderson C, Bain C. Interpreting the evidence choosing between randomised and nonrandomised studies. BMJ 1999;319:312e5. [12] Benson K, Hartz AJ. A comparison of observational studies and randomized controlled trials. N Engl J Med 2000;342:1878e86. [13] Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 2000;342:1887e92. [14] CASS principal investigators and their associates. Coronary artery surgery study (CASS): a randomized trial of coronary bypass surgery. Comparability of entry characteristics and survival in randomized patients and nonrandomized patients meeting randomization criteria. J Am Coll Cardiol 1984;3:114e28. [15] Horwitz RI, Viscoli CM, Clemens JD, Sadock RT. Developing improved observational methods for evaluating therapeutic effectiveness. Am J Med 1990;89:630e8. [16] Grodstein F, Stampfer MJ, Manson JE, Colditz GA, Willett WC, Rosner B, et al. Postmenopausal estrogen and progestin use and the risk of cardiovascular disease. N Engl J Med 1996;335:453e61. [17] Grodstein F, Manson JE, Colditz GA, Willett WC, Speizer FE, Stampfel MJ. A prospective, observational study of postmenopausal hormone therapy and primary prevention of cardiovascular disease. Ann Intern Med 2000;133:933e41. [18] Stampfer M, Colditz G. Estrogen replacement therapy and coronary heart disease: a quantitative assessment of the epidemiologic evidence. Prev Med 1991;20:47e63.

10 R.L. Tannen et al. / Journal of Clinical Epidemiology 60 (2007) 686e [19] Grady D, Rubin SB, Pettiti DB, Fox CS, Black D, Ettinger B, et al. Hormone therapy to prevent disease and prolong life in postmenopausal women. Ann Intern Med 1992;117:1016e37. [20] Barrett-Conner E, Grady D. Hormone replacement therapy, heart disease, and other considerations. Annu Rev Public Health 1998;19:55e72. [21] Writing Group for the Women s Health Initiative Investigators. Risks and benefits of estrogen plus progestin in health postmenopausal women. Principal results from the Women s Health Initiative randomized controlled trial. JAMA 2002;288:321e33. [22] Manson JE, Hsia J, Johnson KC, Rossouw JE, Assaf AR, Lasser NL, et al. Estrogen plus progestin and the risk of coronary heart disease. N Engl J Med 2003;349:523e34. [23] Hulley S, Grady D, Bush T, Furberg C, Herrington D, Riggs B, et al. Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. JAMA 1998;280:605e13. [24] Grady D, Herrington D, Bittner V, Blumenthal R, Davidson M, Hlatky M, et al. Cardiovascular disease outcomes during 6.8 years of hormone therapy. Heart and estrogen/progestin replace study follow-up (HERS II). JAMA 2002;288:49e57. [25] Tannen RL, Weiner MG, Marcus SM. Simulation of the Syst-Eur randomized control trial using a primary care electronic medical record was feasible. J Clin Epidemiol 2006;59:254e64. [26] Gelfand JM, Margolis DJ, Dattani H. The UK General Practice Research Data Base. In: Strom BL, editor. Pharmacoepidemiology. 4th ed.. New York, NY: John Wiley and Sons; p. 337e46. [27] Weiner MG, Karnhart K, Xie D, Tannen RL. Hormone replacement in early menopause does not prevent coronary heart disease. Results of a novel, validated observational study design. (Submitted for publication). [28] Raghunathan TE, Solenberger PW, Van Hoewyk J. IVEware: imputation and variance estimation software. Ann Arbor, MI: Institute for Social Research, University of Michigan; [29] Rubin D. Multiple imputation for non-response in surveys. New York, NY: John Wiley and Sons, Inc; [30] Raghunathan TE, Lepkowski JM, Van Hoewyk J, Solenberger PW. A multivariate technique for multiply imputing missing values using a sequence of regression models. Survey Methodol 2001;27: 85e95. [31] Rosenbaum P, Rubin D. The central role of the propensity score in observational studies for causal effects. Biometrika 1983;70:41e55. [32] Rosenbaum P, Rubin D. Reducing bias in observational studies using subclassification on the propensity score. J Am Stat Assoc 1984;79: 516e24. [33] D Agostino RB, Rubin DB. Estimating and using propensity scores with partially missing data. J Am Stat Assoc 2000;95:749e59. [34] Grodstein F, Clarkson TB, Manson JE. Understanding the divergent data on postmenopausal hormone therapy. N Engl J Med 2003;348: 645e50. [35] Michels KB, Manson JE. Postmenopausal hormone therapy: a reversal of fortune. Circulation 2003;107:1830e3. [36] Hernan MA, Robins JM, Garcia-Rodriguez LA. Discussions on statistical issues arising in the Women s Health Initiative. Biometrics 2005;61:922e30. [37] Million Women Study Collaborators. Patterns of use of hormone replacement therapy in one million women in Britain, 1996e2000. BJOG 2002;109:1319e30. [38] Humphrey LL, Chan BKS, Sox HC. Postmenopausal hormone replacement therapy and the primary prevention of cardiovascular disease. Ann Intern Med 2002;137:273e84. [39] Stahlberg C, Pederson AT, Lynge E, Ottesen B. Hormone replacement therapy and risk of breast cancer: the role of progestins. Acta Obstet Gynecol Scand 2003;82:335e44. [40] Catherino WH, Jordan VC. Nomegestrol acetate, a clinically useful 19-norprogesterone derivative which lacks estrogenic activity. J Steroid Biochem Mol Biol 1995;55:239e46. [41] Archer DF, Navarro FJ, Leslie S, Mirkin S. Effects of levonorgestrel, medroxyprogesterone acetate, norethindrone, and 17beta-estradiol on vascular endothelial growth factor isomers 121 and 165 in Ishikawa cells. Fertil Steril 2004;81:165e70. [42] Kim CJ, Min YK, Ryu WS, Kwak JW, Ryoo UH. Effect of hormone replacement therapy on lipoprotein(a) and lipid levels in postmenopausal women. Influence of various progestogens and duration of therapy. Arch Intern Med 1996;156:1693e700. [43] Register TC, Adams MR, Golden DL, Clarkson TB. Conjugated equine estrogens alone, but not in combination with medroxyprogesterone acetate, inhibit aortic connective tissue remodeling after plasma lipid lowering in female monkeys. Arterioscler Thromb Vasc Biol 1998;18:1164e71. [44] Bladbjerg EM, Skouby SO, Andersen LF, Jespersen J. Effects of different progestin regimens in hormone replacement therapy on blood coagulation factor VII and tissue factor pathway inhibitor. Hum Reprod 2002;17:3235e41. [45] Manson JE, Martin KA. Clinical practice. Postmenopausal hormone replacement therapy. N Engl J Med 2001;354:34e40. [46] Ray WA. Evaluating medication effects outside of clinical trials: new-user designs. Am J Epidemiol 2003;158:915e20. [47] Prentice RL, Langer R, Stefanick ML, Howard BV, Pettinger M, Anderson G, et al. Combined postmenopausal hormone therapy and cardiovascular disease: toward resolving the discrepancy between observational studies and the Women s Health Initiative clinical trial. Am J Epidemiol 2005;162:404e14. [48] Varas-Lorenzo C, Garcia-Rodriguez LA, Perez-Gutthann S, Duque-Oliart A. Hormone replacement therapy and incidence of acute myocardial infarction. A population-based nested case-control study. Circulation 2000;101:2572e8. [49] Kim J, Evans S, Smeeth L, Pocock S. Hormone replacement therapy and acute myocardial infarction: a large observational study exploring the influence of age. Int J Epidemiol 2006;35:731e8.