Definitions of antiretroviral treatment failure for measuring quality outcomes

DOI: 10.1111/j.1468-1293.2009.00808.x r 2010 British HIV Association HIV Medicine (2010), 11, 427 431 ORIGINAL RESEARCH Definitions of antiretroviral treatment failure for measuring quality outcomes A Samaranayake, 1,2,3 MY Chen, 1,2 J McNeil, 4 TRH Read, 1,2 JS Hocking, 5 CS Bradshaw 1,4 and CK Fairley 1,2 1 Melbourne Sexual Health Centre, Melbourne, Australia, 2 School of Population Health, University of Melbourne, Melbourne, Australia, 3 Ministry of Health, Colombo, Sri Lanka, 4 Department of Epidemiology & Preventive Medicine, Monash University, Melbourne, Australia and 5 Key Centre for Women s Health in Society, Melbourne, Australia Objectives Our aim was to compare three different definitions of treatment failure and discuss their use as quality outcome measures for a clinical service. Methods Data for treatment-naïve patients who attended the Melbourne Sexual Health Centre (MSHC) between 1 January 2000 and 31 December 2008 were analysed. Definition 1 was the strict Food and Drug Administration (FDA) definition of treatment failure as determined using the time to loss of virological response (TLOVR) algorithm. Definition 2 defined treatment failure as occurring in those whose viral load never fell to o400 HIV-1 RNA copies/ml or who developed two consecutive viral loads 400 copies/ml on any treatment (switching or stopping treatment with a viral load o400 copies/ml was permitted). Definition 3 was the same as definition 2 except that individuals were also deemed to have failed if they stopped treatment for 6 months or longer. Results There were 310 antiretroviral-naïve patients who started treatment in the study period. Of these, 156 [50.3%; 95% confidence interval (CI) 42.1 53.3%] experienced treatment failure under definition 1, 10 (3.2%; 95% CI 1.5 5.8%) experienced treatment failure under definition 2, and 16 (4.5%; 95% CI 2.5 7.4%) experienced treatment failure under definition 3 over the 108 months of follow-up. The probability of failing definition 1 was statistically different from the probability of failing definition 2or3(P50.01). Conclusion There were significant differences in treatment failure for the three definitions. If definition 1 were used, the outcomes would be sufficiently common to enable clinics to be compared but would be less meaningful. If definition 2 or 3 were used, the events would be too rare to enable clinics to be compared, but it would be possible to set a benchmark level of success that clinics could aim to reach. Keywords: HIV, quality measure, treatment failure Accepted 20 October 2009 Introduction Increasingly, clinical services are required to report on the quality of the care they provide [1]. This commonly involves the reporting of process indicators, that is, whether certain actions have occurred; for example, the Correspondence: Professor Christopher K. Fairley, Melbourne Sexual Health Centre, School of Population Health, University of Melbourne, 580 Swanston Street, Carlton, Vic 3053, Australia. Tel: 1 61 3 9341 6236; fax: 1 61 3 9347 6757; e-mail: cfairley@unimelb.edu.au proportion of patients with acute myocardial infarction given aspirin at arrival [2 4]. Clinical services are also reporting on outcome indicators (e.g. 30-day mortality after myocardial infarction) [2]. Currently, there are no recommendations on the clinical outcome indicators that clinical services for patients with HIV should use. Opportunistic infections and death are now rare events among patients diagnosed with HIV infection in developed countries, making these less relevant outcomes [5]. A single paper has looked at seven process indicators 427

428 A Samaranayake et al. and one outcome measure among HIV-infected patients [2]. These eight indicators were chosen from the US and European HIV treatment guidelines. The outcome indicator that was used was the virological response to HIV treatment, but purely measured as the proportion of patients at the last visit who had an HIV viral load of o400 HIV-1 RNA copies/ml. A more complex analysis of the virological response to HIV treatment is used by the US Food and Drug Administration (FDA) for clinical trials comparing the outcomes of two different treatment regimens [6]. There has, however, been little discussion in the literature about how best to measure virological response as a quality indicator, because the main use to date for this variable has been to compare the efficacies of different antiretroviral regimens. If an outcome indicator is to be useful for a measure of quality in clinical practice, it should fulfil a number of requirements in addition to correlating well with the patients future prognosis [4]. These characteristics include the ease and feasibility of collection and the degree to which the outcomes are predicted by differences in the provider characteristics rather than differences among individual patients. Our aim in this study was to describe the HIV virological response for a single health service using three different definitions of treatment failure and to discuss their relationship to the requirements of a quality outcome measure. We included three measures of virological response, including the definition recommended by the US FDA, called the time to loss of virologic response (TLOVR) algorithm [6]. Methodology The clinical data for this study were obtained for HIVinfected patients attending the Melbourne Sexual Health Centre between January 2000 and December 2008. During this period, 310 HIV-positive patients commenced antiretroviral treatment for the first time (i.e. were antiretroviral naïve). The electronic medical record data, including laboratory measures and HIV treatment histories for each patient, were examined. Clinical files were reviewed to determine the reason for any change in HIV treatment. The outcomes of treatment were assessed using a number of different definitions of treatment failure. In the first analysis (definition 1), we used the TLOVR algorithm, where an individual is deemed to have failed if a plasma HIV-1 RNA level o400 copies/ml was never achieved, or they had confirmed virological rebound from o400 copies/ ml on two consecutive readings, or they had discontinued their first treatment regimen for any reason [6]. In the second analysis (definition 2), an individual was deemed to have failed if the plasma HIV-1 RNA was never below 400 copies/ml, or their viral load rebounded above 400 copies/ml (on two consecutive readings) while on any treatment. They were permitted to change treatment so long as their viral load remained below 400 copies/ml and were also permitted to stop treatment as long as their last viral load on treatment was below 400 copies/ml. In the third analysis (definition 3), an individual was deemed to have failed as for definition 2 but this definition included ceasing therapy and not restarting for a period of more than 6 months. We chose to include the TLVOR algorithm (definition 1) because it is commonly used in clinical trials. We also included the second definition because virological failure (excluding treatment changes because of side effects) leads to resistance mutations which correlate with poor prognosis [7,8] and reduce future treatment options. Also, the development of a viral load of 4400 copies/ml on treatment may reflect poor adherence to treatment, which may in turn reflect suboptimal clinical care. In definition 2, changing treatment because of side effects, patient preference for once-daily therapy or other reasons not associated with a detectable viral load was not deemed to be failure, because this was unlikely to lead to the development of resistance. In definition 3, we included patients as having experienced failure if they stopped any treatment for longer than 6 months, because studies have shown that individuals who stop treatment for longer than 6 months have worse outcomes than those who remain on treatment [9]. We then compared the three definitions of failure using Kaplan Meier survival analyses over the study period. In addition, we compared each of the three definitions against itself for two time periods, the first time period being January 2000 to June 2004 and the second being July 2004 to December 2008, to determine if there were significant changes between these periods for the different definitions. Finally, we examined how closely each of these three definitions of treatment failure correlated with the requirements of quality outcome measures. These include: the ease and feasibility of collection of the outcome, the degree to which the outcomes are correlated with the clinical prognosis, the degree to which the outcomes are predicted by differences in the provider characteristics rather than differences among individual patients, the frequency with which an event occurs, and finally the need for risk adjustment before the results can be interpreted [3,4]. Viral load measurements were performed at the Victorian Infectious Diseases Laboratory (VIDRL) using the Roche Amplicor HIV Monitor Version 1.5 (Roche Molecular Diagnostics, Pleasanton, California, USA) UltraSensitive assay for measurement of viral RNA. T-cell lymphocytes (CD4) were measured using flow cytometry.

Measuring quality outcomes of ART 429 Statistical analysis Each endpoint was analysed using a Kaplan Meier survival analysis in SPSS version 17 (SPSS Inc., Chicago, IL, USA). Individuals who had not reached an endpoint by the time of their last viral load measurement were censored. Log rank (Mantel Cox) w 2 was used to determine the significance of differences between definitions and between the two time periods for the same definition. Results There were 310 patients who commenced highly active antiretroviral therapy (HAART) for the first time during the study period. Of these, 268 were male, 41 were female and one was transgender. The median age of the patients was 34 (range 25 70) years. Only 19 (6%) were injecting drug users. At the commencement of antiretroviral treatment, the median viral load was 66100 copies/ml and 84 patients (27%) had a viral load above 100 000 copies/ml. The median CD4 lymphocyte count was 420 cells/ml and 74 patients (24%) had CD4 counts of o200 cells/ml. The outcomes of treatment are shown in Table 1. Of the 310 patients, 156 [50.3%; 95% confidence interval (CI) 42.1 53.3%] experienced treatment failure under definition 1, 10 (3.2%; 95% CI 1.5 5.8%) experienced treatment failure under definition 2, and 16 (4.5%; 95% CI 2.5 7.4%) experienced treatment failure under definition 3 over the 108 months of follow-up. Figure 1 shows the Kaplan Meier analysis of the proportion of individuals who would have been deemed to Table 1 Treatment outcomes in 310 HIV-infected individuals who commenced their initial antiretroviral regimen between January 2000 and December 2008 Outcome Number (%) (N 5 310) Reason for ceasing treatment 1. Continued on initial treatment and viral 154 (49.7) load consistently o400 copies/ml until final follow-up 2. Changed initial treatment and viral load remained o400 copies/ml 140 (45.1) Adverse event (44) Patient s request (10) Other (86) 3. Did not achieve viral load o400 copies/ml 7 (2.2) on any treatment 4. Achieved viral load o400 copies/ml on initial treatment and then developed viral load 4400 copies/ml while on treatment 3 (0.9) 5. Achieved viral load o400 copies/ml on any treatment and stopped treatment for more than 6 months Total number of treatment-naïve patients 310 6 (1.9) Adverse events (3) Patient request (3) Fig. 1 Kaplan Meier analysis of the proportion of individuals who would have been deemed to have failed treatment on the basis of different definitions of failure for the period 2000 2008 at the Melbourne Sexual Health Centre (MSHC). TLOVR, time to loss of virological response. have experienced treatment failure on the basis of the three different definitions. There was a significant difference (P 5 0.01) in the probability of failure between definitions 1 and 2 and between definitions 1 and 3 (P 5 0.01), but not between definitions 2 and 3 (P 5 0.5). To determine whether any definition could show a significant reduction in treatment failure over time, we compared treatment failure during the first half of the study period (2000 mid-2004) with that during the second half (mid-2004 2008) for each of the three definitions separately (Fig. 2a c). Treatment failure was different between the two time periods only for definition 1 (P 5 0.5), and not for either definition 2 (P 5 0.5) or definition 3 (P 5 0.5). Table 2 shows the comparison of the three different definitions for assessing virological response with the characteristics of an ideal quality measure. Discussion We compared three definitions of HIV treatment failure in a single clinical service and compared them with the characteristics of ideal quality outcome measures. The striking observation was that the failure rate was very much higher for the definition using TLOVR than for the other definitions because ceasing treatment for any reason is defined as treatment failure in the TLOVR definition. Because individuals most often ceased or changed treatment for reasons other than virological rebound, the TLOVR definition was the least useful representation of clinical prognosis. In contrast, the rate of failure in definitions 2 and 3 was too low to allow detection of meaningful changes over time, even in a large clinic

430 A Samaranayake et al. (a) (b) (c) Fig. 2 Kaplan Meier analysis of the proportion of individuals who would have been deemed to have failed treatment on the basis of (a) definition 1 [the time to loss of virological response (TLOVR)], (b) definition 2 and (c) definition 3. Treatment was started during different periods at the Melbourne Sexual Health Centre (MSHC). ART, antiretroviral therapy. Table 2 Comparison of the three definitions used to assess HIV treatment outcomes against characteristics of an ideal outcome measure for quality Characteristic Ease and feasibility Correlation with clinical prognosis Correlated with provider characteristics rather than patient characteristics Frequency of event Need for risk adjustment Suitability All definitions would require an electronic clinic database with the start and stop dates of medications and viral load measurements In definition 1, a large number of failures are from changing treatment unrelated to increasing viral loads or the development of resistance and therefore are not related to prognosis. In definition 2, all failures are likely to adversely affect future treatment. In definition 3, prolonged periods off treatment are likely to double the risk of AIDS or death [9] In definition 1, 140 (90%) of the 156 failures were attributable to patients stopping treatment for a variety of reasons. Some reasons (adverse effects) were beyond the providers control but others relating to changing to simpler regimens would have been related to providers and may even reflect good quality care rather than a treatment failure. In definition 2, a proportion of the failures would have been attributable to suboptimal support for adherence (provider characteristic) but others would have been attributable to patients at higher risk of failure (high viral load, mental illness or injecting drug use) [5] With definition 1, 156 individuals experienced treatment failure compared with only 10 individuals with definition 2 and 16 with definition 3; the last two definitions therefore lack statistical power to detect a difference All three definitions would require risk adjustment for factors associated with a higher risk of failure. However, in definition 1, because many of the failures are attributable to factors related to ceasing medication for reasons other than an increased viral load, the adjustment would be less meaningful Definition 1 is the time to loss of virological response (TLOVR). In definition 2, an individual was deemed to have experienced treatment failure if the plasma HIV-1 RNA was never o400 copies/ml, or their viral load rebounded above 400 copies/ml while on the first regimen. In definition 3, an individual was deemed to have experienced treatment failure if the plasma HIV-1 RNA was never o400 copies/ml, or rebounded above 400 copies/ml on the first regimen or if they stopped all treatment for more than 6 months for any reason. service such as ours. No single definition stood out as superior for the other requirements of a quality outcome measure. This is the first study to assess different definitions of HIV treatment failure and to compare these with the requirements used to evaluate quality outcome measures in a single health service. On the basis of these findings, it may be that the best option is to set a benchmark level for either definition 2 or definition 3 and to monitor it to ensure that it remains high. This study has a number of limitations that should be considered when evaluating these data. Firstly, there are no internationally accepted requirements for assessing a quality outcome measure. Those presented in this paper have been taken from authoritative reviews in the

Measuring quality outcomes of ART 431 literature and are generally accepted as important characteristics. Secondly, we have used a viral load of o400 copies/ml rather than o50 copies/ml. We did this because this sort of analysis requires historical data and viral loads at the laboratory were not always reported as o50 copies/ml. In our study, only definition 1 was able to detect a significant difference in treatment failure between the earlier 4.5-year time period and the later 4.5-year time period. No difference was apparent between these two time periods when either definition 2 or definition 3 was used, as failure was a rare outcome for both of these definitions. Given that definitions 2 and 3 are more strongly correlated with prognosis than definition 1, it is unlikely that the statistical difference detected was not clinically important. We would argue that perhaps the most important requirement of a quality measure is that it relates to the patient s prognosis. However, given that failure according to definitions 2 and 3 is now quite uncommon, it will not occur sufficiently often to enable the detection of sizable differences in failure within the same clinic over time or between different clinics. We would therefore argue that these definitions should not be used to compare different clinical services but that perhaps an internationally agreed standard that is adjusted for the risk profile of patients is agreed upon. We are presenting these data to encourage international discussion on how to monitor quality of HIV care and we propose that reporting rates of virological failure is the most practical and meaningful way of doing this. We conclude by asking whether we need a benchmark minimum level of virological failure that includes appropriate risk adjustment. References 1 Rosen AK, Rivard P, Zhao S et al. Evaluating the patient safety indicators: how well do they perform on Veterans Health Administration data? Med Care 2005; 43: 873 884. 2 Wilson IB, Landon BE, Marsden PV et al. Correlations among measures of quality in HIV care in the United States: cross sectional study. BMJ 2007; 335: 1085. 3 Pronovost PJ, Nolan T, Zeger S, Miller M, Rubin H. How can clinicians measure safety and quality in acute care? Lancet 2004; 363: 1061 1067. 4 Mainze J. Methodology matters defining and classifying clinical indicators for quality improvement. Int J Qual Health Care 2003; 15: 523 530. 5 Powderly WG, Saag MS, Chapman S, Yu G, Quart B, Clendeninn NJ. Predictors of optimal virological response to potent antiretroviral therapy. AIDS 1999; 13: 1873 1880. 6 US Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER). Guidance for Industry. Antiretroviral Drugs Using Plasma HIV RNA Measurements Clinical Considerations for Accelerated and Traditional Approval, October 2002. Available at www.fda.gov/cder/guidance/index.htm. 7 Cozzi-Lepri A, Phillips AN, Clotet B et al. for the Euro-SIDA Study Group. Detection of HIV drug resistance during antiretroviral treatment and clinical progression in a large European cohort study. AIDS 2008; 22: 2187 2198. 8 Kozal MJ, Hullsiek KH, Macarthur RD et al. TheIncidenceofHIV drug resistance and its impact on progression of HIV disease among antiretroviral-naïve participants started on three different antiretroviral therapy strategies. HIV Clin Trials 2007; 8: 357 370. 9 SMART Study, National institute of allergy and infectious diseases. N Engl J Med 2006; 355: 2283 2296.