Yangxin Huang a & Hulin Wu b a Department of Epidemiology & Biostatistics, College of Public

Transcription

1 This article was downloaded by: [University of Prince Edward Island] On: 15 April 2013, At: 05:06 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: Registered office: Mortimer House, Mortimer Street, London W1T 3JH, UK Journal of Applied Statistics Publication details, including instructions for authors and subscription information: A Bayesian approach for estimating antiviral efficacy in HIV dynamic models Yangxin Huang a & Hulin Wu b a Department of Epidemiology & Biostatistics, College of Public Health, University of South Florida, Tampa, Florida, USA b Department of Biostatistics & Computational Biology, University of Rochester School of Medicine and Dentistry, Rochester, New York, USA Version of record first published: 19 Aug To cite this article: Yangxin Huang & Hulin Wu (2006): A Bayesian approach for estimating antiviral efficacy in HIV dynamic models, Journal of Applied Statistics, 33:2, To link to this article: PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

2 Journal of Applied Statistics Vol. 33, No. 2, , March 2006 A Bayesian Approach for Estimating Antiviral Efficacy in HIV Dynamic Models YANGXIN HUANG & HULIN WU Department of Epidemiology & Biostatistics, College of Public Health, University of South Florida, Tampa, Florida, USA, Department of Biostatistics & Computational Biology, University of Rochester School of Medicine and Dentistry, Rochester, New York, USA ABSTRACT The study of HIV dynamics is one of the most important developments in recent AIDS research. It has led to a new understanding of the pathogenesis of HIV infection. Although important findings in HIV dynamics have been published in prestigious scientific journals, the statistical methods for parameter estimation and model-fitting used in those papers appear surprisingly crude and have not been studied in more detail. For example, the unidentifiable parameters were simply imputed by mean estimates from previous studies, and important pharmacological/clinical factors were not considered in the modelling. In this paper, a viral dynamic model is developed to evaluate the effect of pharmacokinetic variation, drug resistance and adherence on antiviral responses. In the context of this model, we investigate a Bayesian modelling approach under a non-linear mixed-effects (NLME) model framework. In particular, our modelling strategy allows us to estimate time-varying antiviral efficacy of a regimen during the whole course of a treatment period by incorporating the information of drug exposure and drug susceptibility. Both simulated and real clinical data examples are given to illustrate the proposed approach. The Bayesian approach has great potential to be used in many aspects of viral dynamics modelling since it allow us to fit complex dynamic models and identify all the model parameters. Our results suggest that Bayesian approach for estimating parameters in HIV dynamic models is flexible and powerful. KEY WORDS: dynamics Bayesian mixed-effects models, drug efficacy, drug resistance, HIV, MCMC, viral Introduction Quantitative studies of plasma HIV RNA during treatment with potent anti-retroviral (ARV) therapies have provided crucial information on drug efficacy and mechanisms of HIV/AIDS pathogenesis. HIV dynamic models can provide theoretical principles to guide the development of treatment strategies for HIV-infected patients (Ho et al., 1995; Huang et al., 2003; Nowak et al., 1995; Perelson, 1989; Perelson et al., 1996, 1997; Wei et al., 1995; Wu & Ding, 1999). Although important findings in HIV dynamics Correspondence Address: Yangxin Huang, Department of Epidemiology and Biostatistics, College of Public Health, University of South Florida, Bruce B. Downa Blvd., DMC 56, Tampa, FL , USA. yhuang@hsc.usf.edu Print= Online=06= # 2006 Taylor & Francis DOI: =

3 156 Y. Huang & H. Wu have been published in prestigious scientific journals (Ho et al., 1995; Perelson et al., 1996, 1997; Wei et al., 1995), the statistical methods used in those papers appear to be very crude. Linear and non-linear models were fitted to plasma HIV-1 RNA data from patients on an individual basis. The fitted viral dynamic parameters were then summarized over the population in average and standard deviation. Recently there has been great interest in estimating antiviral efficacy and HIV dynamic parameters in order to acquire more comprehensive understanding of the pathogenesis of HIV infection and to assess the potency of ARV treatment (Han et al., 2002; Perelson et al., 1996, 1997; Putter et al., 2002; Wu et al., 1998; Wu & Ding, 1999). The structure of data in viral dynamic studies is that of repeated viral load measurements on each of a number of subjects. Because the viral dynamic process described by the models takes place within a given subject and the process may differ among subjects, the hierarchical non-linear mixed-effects model approach for HIV studies has been initially proposed by Wu et al. (1998) and Wu & Ding (1999) although the techniques have been used in modelling pharmacokinetics/pharmacodynamics (PK/PD) for many years (Beal & Sheiner, 1985). Mixed-effects models offer a flexible framework where dynamic parameters for both individuals and population can be estimated by combining information across all subjects. The level of computational complexity for obtaining the estimates in non-linear mixed-effects (NLME) models is considerable (Wu et al., 1998; Wu & Ding 1999). The NLME model fitting can be implemented in standard statistical software, such as the function nlme() in S-plus (Pinheiro & Bates, 2000), the procedure NLMIXED in SAS (2000) and other packages like NONMEM (Boeckmann et al., 1989) and NLMEM (Galecki, 1998). However, a factor arguing against using NLME models is the fact that in the case of viral dynamic modelling we may not have an explicit closedform function to fit to. For example, our non-linear function may be a solution (to be determined numerically) of a system of non-linear ordinary differential equations. It is possible to dynamically load FORTRAN code into S-plus, but it was found that the combination nlme() and FORTRAN code is too unstable and too intensive to be a competitive option (Putter et al., 2002). Bayesian statistics has made great strides in recent years. A class of methods have been developed for estimation and inference via stochastic simulation known as Markov chain Monte Carlo (MCMC) methods which are widely applied in various fields (Bennett et al., 1996; Gamerman, 1997; Gelfand et al., 1990; Gelman et al., 1996; Han et al., 2002; Lunn et al., 2002; Putter et al., 2002; Raftery & Lewis, 1992; Wakefield et al., 1994; Wakefield, 1996). In particular, the MCMC approach has been introduced in non-linear mixed-effects models (NLME) with applications in PK/PD modelling since mid-1990s (Bennett et al., 1996; Gelman et al., 1996; Lunn et al., 2002; Wakefield et al., 1994; Wakefield, 1996). Bayesian analysis for a population HIV dynamic model was investigated by Putter et al. (2002) and Han et al. (2002) whose studies were based on the early dynamics of post-treatment HIV-RNA decline. Han et al. considered a simple model with constant uninfected target cells so that viral load has an explicit function form. Although Putter et al. (2002) extended the model to the system of non-linear differential equations without analytic solution, they estimated the parameters from data that were taken no more than 2 weeks after initiating antiviral treatment and a constant drug efficacy was assumed. In addition, they did not recognize the fact of variability in drug susceptibility (due to drug resistance) and adherence in the presence of ARV therapy. They also require frequent data in the early stage of treatment. This paper presents a Bayesian modelling approach to parameter estimation in an HIV dynamic model with a long-term ARV treatment, which is a system of non-linear differential equations with time-varying coefficients. In particular, we consider that antiviral

4 Estimating Antiviral Efficacy in HIV Dynamic Models 157 efficacy may vary during treatment due to a variation of drug concentrations, the change of drug susceptibility, and unpredictable adherence behaviour. In our model fitting, longterm viral load data (including viral rebound data) can be used and very frequent measurements during the early stage of treatment may not be necessary. In the next section, a simplified viral dynamic model with time-varying drug efficacy is suggested to describe the antiviral response. The section after describes a drug efficacy model, in which the effect of pharmacokinetic (PK) variation, drug resistance and adherence are incorporated. A Bayesian non-linear mixed-effects modelling approach for estimating parameters, in particular antiviral efficacy parameters, in HIV dynamic models is proposed in the fourth section. A numerical example from a simulated dataset is presented to illustrate our approach and the proposed methodology is applied to an AIDS clinical trial study to estimate dynamic parameters in the fifth section, and finally, we conclude the paper with some discussions in the sixth section. HIV Dynamic Model The basic dynamic model is a model that describes the population dynamics of HIV and its target cells in plasma. The mathematical models for HIV dynamics have been applied to clinical data, and these applications have recently resulted in very important findings on the pathogenesis of HIV infection (Ho et al., 1995; Perelson et al., 1993, 1996, 1997; Perelson & Nelson, 1999; Wei et al., 1995; Wu & Ding, 1999). A simplified model for HIV dynamics after initiation of ARV treatment (Nowak & May, 2000) can be written as d T ¼ l rt ½1 g(t)šktv dt d dt T ¼½1 g(t)šktv dt d dt V ¼ NdT cv where T, T and V are target uninfected cells, infected cells and virus, respectively, l represents the rate at which new T cells are created from sources within the body, such as the thymus, r is the death rate per T cell, k is the rate at which T cells become infected by virus, d is the rates of death for infected cells, N is the number of new virions produced from each of infected cells during their life-time, and c is the death (clearance) rate of free virions. The death rate of free virions (c) is greater than that of infected cells (d). The time-varying parameter g (t) is the antiviral drug efficacy, as defined in the next section. The model is both simple and adequate to describe the HIV population response to ARV treatment although more complicated models (Nowak & May, 2000; Perelson & Nelson, 1999) are also available. We assume that the system is in a steady-state before initiating ARV treatment. Thus the initial conditions for the system of equations (1) are (1) T 0 ¼ c kn, T 0 ¼ cv 0 dn, V 0 ¼ ln c r k (2) If the drug is not 100% effective (not perfect inhibition), the system of ordinary differential equations cannot be solved analytically. The solutions to equation (1) then have to be evaluated numerically. Let b ¼ (f, c, d, l, r, N, k) T denote a set of parameter values,

5 158 Y. Huang & H. Wu where f is a parameter in the drug efficacy model (see the next section for details). In the estimation procedure, we only need to evaluate the difference between observed data and numerical solutions of V(t). So there is no need for an explicit solution of equation (1). Similar to the analysis in Huang et al. (2003), it can be shown from equation (1) that if drug efficacy g (t). e (e is a threshold of drug efficacy) for all t, where e ¼ 1 cr knl (3) the dynamic system (1) converges to a stable uninfected steady-state and the virus will be eventually eradicated. However, if g (t), e (treatment is not potent enough) or if the potency falls below e before virus eradication (due to drug resistance, for example), the uninfected state is not stable and the endemically infected state exists. In the latter situation, viral load may rebound; see Huang et al. (2003) for a detailed discussion. Antiviral Drug Efficacy Within the population of HIV virions in a human host, there is likely to be genetic diversity and corresponding diversity in sensitivity to the various ARVs. In some clinical studies, genotype or phenotype tests are conducted to determine the sensitivity of ARV agents before a treatment regimen is selected. Here we use the phenotype marker IC 50 (Molla et al., 1996), which represents the drug concentration necessary to inhibit viral replication by 50%, to quantify agent-specific drug sensitivity. Herein we refer to this quantity as the median inhibitory concentration. To model the within-host changes over time in IC 50 due to the emergence of new drug resistant mutations, we use the following function (Huang et al., 2003), 8 < IC 50 (t) ¼ I 0 þ I r I 0 t for 0, t, t r, t : r I r for t t r, where I 0 and I r are respective values of IC 50 (t) at baseline and time point t r at which the resistant mutations dominate. If I r ¼ I 0, no new mutation is developed during treatment. An example of such a function over the first 30 days is plotted in Figure 1(a). Although more complicated models for median inhibitory concentration have been proposed based on the frequencies of resistant mutations (likely to arise from the regimen) and (4) Figure 1. Courses of median inhibitory concentration (IC 50 ) and drug efficacy over time. (a): IC 50 curve with I 0 ¼ 0:1, I r ¼ 15 and t r ¼ 20 days. (b): The time-course of drug efficacy with f ¼ 1, C min ¼ 45 and A(t) ¼ 1 (for all t) affected by IC 50 time-course displayed panel in (a)

6 Estimating Antiviral Efficacy in HIV Dynamic Models 159 cross-resistance patterns (Bonhoeffer et al., 1997), in clinical studies it is common to measure IC 50 values only at baseline and failure time (Molla et al., 1996). Thus, this function may serve as a good first approximation. Poor adherence to a treatment regimen is one of the major causes of treatment failure (Besch, 1995). Patients may miss occasional doses, may misunderstand prescription instructions or may miss multiple consecutive doses for various reasons; these deviations from prescribed dosing affect drug exposure in predictable ways. Thus, the effect of adherence is defined as A(t) ¼ 1 for i< t, (i þ 1)< and dose is taken at i< 0 for i< t, (i þ 1)< and dose is missed at i< (5) where < denotes the recommended dosing interval. In recent years, a number of ARV drugs have been developed. Three types of ARV agents, nucleoside/non-nucleoside reverse transcriptase inhibitors (RTI) and protease inhibitors (PI) have been widely used in developed countries. Highly active antiretroviral therapy (HAART), containing a combination of drugs that inhibit the replication of HIV, has proven to be extremely effective at reducing the amount of virus in the blood and tissues of infected patients. In viral dynamic studies by Wu & Ding (1999), Perelson & Nelson (1999), Ding & Wu (1999), investigators assumed that the drug efficacy was constant over treatment time. However, drug efficacy may vary as the concentrations of ARV drugs and other factors (e.g. drug resistance) vary during treatment (Ding & Wu, 1999; Perelson & Nelson, 1999; Wahl & Nowak, 2000). Also in practice, patients viral load may rebound during treatment and the rebound may be associated with resistance to ARV therapy (Wahl & Nowak, 2000). Here we use the following modified E max model (Sheiner, 1985) for the drug efficacy of the ARV agent, g (t) ¼ C min A(t) fic 50 (t) þ C min A(t) ¼ IQ(t)A(t) f þ IQ(t)A(t) where C min is the minimum concentration of drug in plasma and IQ(t) ¼ C min =IC 50 (t) denotes the inhibitory quotient (IQ) which is the PK adjusted phenotypic susceptibility (Hsu et al., 2000). C min can be replaced by time-varying drug concentration (Huang et al., 2003) or other PK parameters, such as area under the plasma concentration-time curve (AUC). Although IC 50 can be measured by phenotype assays in vitro, it may not be equivalent to the IC 50 in vivo. Parameter f indicates a conversion factor between the two. Lack of adherence reduces the drug exposure, which can be quantified by equation (5), and thus reduces drug efficacy based on formula (6) and, in turn, can affect viral response. Time-varying parameter g (t) ranges from 0 to 1 and indicates the drug efficacy (the inhibition rate of viral replication) in a viral dynamic (response) model. Thus if g (t) ¼ 1, the drug is 100% effective, while if g (t) ¼ 0, the drug has no effect. An example of the timecourse of the drug efficacy g (t) with perfect adherence is shown in Figure 1(b). More details regarding viral dynamic models and time-varying drug efficacy models can be found in Huang et al. (2003). Note that, if C min, A(t) and IC 50 (t) can be observed and f can be estimated from clinical data, then time-varying drug efficacy g (t) can be estimated during the course of antiviral treatment. Also note that we incorporate drug resistance and drug exposure directly in the drug efficacy parameters based on principles of mass action, instead of modelling the quasi-species of virus mutations (Nowak & May, 2000). (6)

7 160 Y. Huang & H. Wu Bayesian Modelling and Parameter Estimation Bayesian Modelling Approach Although the non-linear least squares (NLS) method is a simple and popular approach to estimate parameters in a non-linear system, the parameters for each subject need to be estimated independently, which requires more data from each subject and may not be efficient. In order to avoid this potential problem, we use an alternative method, Bayesian hierarchical (mixed-effects) modelling approach, for estimating parameters in HIV dynamic models. The attractiveness of a Bayesian hierarchical approach is that an extremely complicated model can be built out of a succession of relatively simple components. Variability is accounted for in a straightforward way, and the resulting posterior distribution can be used to answer many statistical inference questions. Since there are too many (seven) parameters in viral dynamic model (1), not all of them can be identified from clinical data. In many analyses, there are nuisance parameters, those that are of little or no interest to the study at hand. The use of prior distributions allows information on these nuisance parameters to be included in the analysis, allowing complex models to be fitted more reliably to datasets. In order efficiently to use prior information to identify these parameters, the Bayesian approach, which can maximize statistical power by using all the information available, whilst taking into account uncertainty in both the data and parameter estimates, ensures that the prior information is used to avoid the identifiability problem. Thus, a Bayesian approach can be used to estimate all unknown parameters (although some priors are strongly informative) in HIV dynamic model (1). In choosing an optimal parametrization to fit this model, the following two points are considered. First, prior knowledge about a number of parameters can be incorporated in a Bayesian setting. Secondly, it is advantageous to transform the unknown parameters in order to ensure that constraints on parameters are naturally fulfilled (Han et al., 2002; Putter et al., 2002; Wakefield, 1996). Note that throughout this paper, natural logarithmic transformation for parameters will be taken. Denote the number of subjects by n and the number of measurements on the ith subject by m i. In the model (1), we are able to obtain measurements for viral load V(t) from AIDS clinical trials. For notational convenience, let m ¼ ( log f, log c, log d, log l, log r, log N, log k) T, Q ¼ {u i,i ¼ 1,..., n}, u i ¼ ( log f i, i, log d i, log l i, log r i, log N i, log k i ) T, Q {i} ¼ {u l, l = i} and Y ¼ {y ij, i ¼ 1,..., n; j ¼ 1,..., m i }. Let f ij (u i, t j ) be the numerical solution of common logarithmic viral load log 10 (V(t)) to the differential equations (1) for the ith subject at time t j. The repeated measurements of common logarithmic viral load for each subject, y ij (t), at treatment times t j, j ¼ 1, 2,..., m i, can be written as y ij (t j ) ¼ f ij (u i, t j ) þ e i (t j ), i ¼ 1,..., n; j ¼ 1,..., m i (7) where e i (t j ) is a measurement error with mean zero. Then, a Bayesian nonlinear mixedeffects model can be written as following three stages (Davidian & Giltinan, 1995). Stage 1. Within-subject variation in common logarithmic viral load measurements: y i ¼ f i (u i ) þ e i, ½e i js 2, u i Š N(0, s 2 I mi ) (8) where y i ¼ ( y i1 (t 1 ),..., y imi (t mi )) T, f i (u i ) ¼ ( f i1 (u i, t 1 ),..., f imi (u i, t mi )) T, e i ¼ (e i (t 1 ),..., e i (t mi )) T and the bracket notation ½AjBŠ denotes the conditional distribution of A given B.

8 Estimating Antiviral Efficacy in HIV Dynamic Models 161 Stage 2. Between-subject variation: u i ¼ m þ b i, ½b i jsš N(0, S) (9) Stage 3. Hyperprior distribution: s 2 Ga(a, b), m N(h, L), S 1 Wi(V, n) (10) where the mutually independent Gamma (Ga), Normal (N) and Wishart (Wi) prior distributions are chosen to facilitate computations (Davidian & Giltinan, 1995; Gelfand et al., 1990). Note that the parametrization of Gamma and Wishart distribution is such that Ga(a, b) has mean ab and Wi(V, n) has mean matrix nv. The parameters a, b, h, L, V and n that characterize the hyperprior distributions are known. MCMC Implementation Following the studies by Davidian & Giltinan (1995), Gelfand et al. (1990) and Wakefield et al. (1994), it can be shown from equations (8) (10) that the full conditional distributions for the parameters s 2, m and S 1 may be written explicitly as ½s 2 jm, S 1, Q, YŠ Ga a þ Sn i¼1 m i 1, 2 b þ 1! 1 2 Sn i¼1 Sm i j¼1 ½y ij f ij (u i, t j )Š 2 ¼ p(s 2 jm, S 1, Q, Y) ½mjs 2, S 1, Q, YŠ N (ns 1 þ L 1 ) 1 (S 1 S n i¼1 u i þ L 1 h), (ns 1 þ L 1 ) 1 ¼ p(mjs 2, S 1, Q, Y) ½S 1 js 2, m, Q, YŠ Wi V 1 þ S n i¼1 (u i m)(u i m) T 1, n þ n ¼ p(s 1 js 2, m, Q, Y) Here, however, the full conditional distribution of each u i, given the remaining parameters and the data, cannot be calculated explicitly. The distribution of ½u i js 2, m, S 1,Q {i}, YŠ has a density function that is proportional to exp s 2 S m i j¼1 2 ½ y ij f ij (u i, t j )Š (u i m) T S 1 (u i m) ¼ p(u i js 2, m, S 1, Q {i}, Y) It remains to specify the values of the hyper-parameters in the prior distributions (10). In principle, if we have reliable prior information for some of parameters, strong prior (smaller variance) may be used for these parameters. For those other parameters such as f, we may not have enough prior information or we may intend to use the available clinical data to determine these parameters since they are critical to quantify betweensubject variations in response; a non-informative prior (larger variance) may be given for these parameters. In particular, one usually chooses non-informative prior distributions for parameters of interest (Carlin & Louis, 1996).

9 162 Y. Huang & H. Wu We know that Bayesian techniques enable: (i) estimating variances of quantities of interest when asymptotic methods are difficult to apply or inappropriate due to small sample sizes; (ii) fitting complex models using powerful Markov chain Monte Carlo (MCMC) methods. MCMC methods are an established suite of methodologies that enable samples to be drawn from the joint and marginal posterior distributions of unknown quantities in Bayesian models and of functions of the unknowns. The methods work by defining a Markov chain whose stationary density is equal to the target density. The chain is then simulated for a time deemed adequate for convergence to have occurred, and the samples are drawn from the simulated chain. These samples are, provided convergence has occurred, samples from the target density of interest. To implement an MCMC algorithm, here Gibbs sampling steps are used to update s 2, m and S 1, while we update u i, i ¼ 1,..., n, using a Metropolis-Hastings (M-H) step. To implement the M-H algorithm, it is necessary to specify a suitable proposal density. Several possible choices of proposal density are discussed in the literature and a natural choice of M-H algorithm for updating u i is a random-walk chain (Bennett et al., 1996; Carlin & Louis, 1996; Chib & Greenberg, 1995; Gelman et al., 1996; Roberts, 1996). In particular, the proposal density is chosen to be a multivariate normal distribution centred at the current value of u i, since it is easily sampled and clearly symmetric (Carlin & Louis, 1996; Chib & Greenberg, 1995; Gelman et al., 1996; Roberts, 1996; Wakefield, 1996). Note that an important issue is the choice of the dispersion of the proposal density. If the variance of the proposed density is too large, an extremely large proportion of iterations will be rejected, and the algorithm will therefore be extremely inefficient. On the other hand, if the variance of the proposed density is too small, the chain will have a high acceptance rate but will move around the parameter space slowly, again leading to inefficiency (Carlin & Louis, 1996; Chib & Greenberg, 1995; Roberts, 1996). We will consider this issue in the numerical examples. As suggested by Geman & Geman (1984), for example, one long run may be more efficient with considerations of the following two points: (i) a number of initial burn-in simulations are discarded, since from an arbitrary starting point it would be unlikely that the initial simulations came from the stationary distribution targeted by the Markov chain; (ii) one may only save every kth (k being an integer) simulation samples to reduce the dependence among samples used for parameter estimation. We are going to use this strategy in our MCMC implementation. However, in real applications, there are a number of important decisions that have to be made. These include the selection of number of iterations, the spacing between iterations retained for the final analysis, and the number of initial burn-in iterations discarded. We will discuss these practical issues in the numerical illustrations in the next section. The iterative MCMC algorithm is outlined as follows. Step 1. Initialize the iteration counter of the chain j ¼ 1 and start with initial values G (0) ¼ (s 2(0), m (0), S 1(0), Q (0) ) T. Step 2. Obtain a new value G (j) ¼ (s 2( j), m ( j), S 1( j), Q ( j) ) T from G ( j 1) through successive generation of values: 1. s 2( j) p (s 2 ju ( j 1), S 1( j 1), Q ( j 1), Y) m ( j) p (mjs 2( j), S 1( j 1), Q ( j 1), Y) S 1( j) p (S 1 js 2( j), m ( j), Q ( j 1), Y)

10 Estimating Antiviral Efficacy in HIV Dynamic Models For u ( j) i, move the chain to a new value w, generated from the proposal (symmetric) density q(wju ( j 1) i ), from u ( j 1) i. Evaluate the acceptance probability of the move, a(wju ( j 1) i ) given by equation (11). If the move is accepted, u ( j) i ¼ w. Ifitisnot accepted, u ( j) i ¼ u ( j 1) i and the chain does not move. Step 3. Change the counter from j to j þ 1 and return to Step 2 until convergence is reached. Step 2.2 is performed after the generation of an independent uniform quantity u.ifu a, the move is accepted and if u. a, the move is not allowed, where ( ) a(wju ( j 1) p (wjs 2( j), m ( j), S 1( j), Q ( j 1) {i}, Y) i ) ¼ min 1, p (u ( j 1) i js 2( j), m ( j), S 1( j), Q ( j 1) {i}, Y) Note that the choice of the proposal density q is essentially arbitrary, although in practice a careful choice will help the algorithm to move quickly around the parameter space. Due to the symmetry of q, q(wju ( j 1) i ) ¼ q(u ( j 1) i jw) and they are cancelled from fractional expression in equation (11). For u i, we use a multivariate normal proposal distribution centred at the current value with the variance covariance matrix given by an informationtype matrix (Bennett et al., 1996; Gelman et al., 1996; Han et al., 2002; Wakefield, 1996). When convergence is reached, the resulting value G ( j) is a draw from the corresponding density. Thus, as the number of iterations increases, the chain approaches its equilibrium condition. Convergence is then assumed to hold approximately. For a more detailed discussion of the MCMC scheme and the convergence of MCMC methods, consisting of a series of Gibbs sampling and M-H algorithms, please refer to the literature (Bennett et al., 1996; Gelfand et al., 1990; Gelman et al., 1996; Lunn et al., 2002; Wakefield, 1996). A good review on Gibbs sampling and M-H algorithms can also be found in Chapters 5 and 6 in Gamerman (1997). After we collect the final MCMC samples, we are interested in the posterior means or quantiles of functions of the parameters, such as credible intervals and posterior medians. Therefore, the sample generated during a run of the algorithm should adequately represent the posterior distribution of interest so that MCMC estimates of such quantities are approximately correct. Numerical Examples To illustrate the approach proposed in this paper, we present two numerical examples, one is based on simulated data and another is based on a data set from an AIDS clinical trial. Based on the discussion in the previous section, the prior distribution for m was assumed to be N(h, L) with L being a diagonal matrix. Following the idea of Han et al. (2002) for prior construction, as an example we discuss the prior construction for log d. The prior constructions for other parameters are similar and so are omitted here. Ho et al. (1995) reported viral dynamic data on 20 patients; the logarithm of the average death rate of infected cells (log d) is Wei et al. (1995) used two different models with a group of 22 subjects to estimate the death rate of infected cells and obtained log d with and 21.33, respectively. Following the studies by Ho et al. (1995) and Wei et al. (1995), Nowak et al. (1995) estimated log d ¼ based on 11 subjects with one possible outlying subject excluded. It can be seen that four estimates of log d from these studies (Ho et al., 1995; Nowak et al., 1995; Wei et al., 1995) are , 20.84, and , respectively. The individual estimates of log d from these studies approximately follow a symmetric normal distribution. Thus we chose a normal distribution N (11)

11 164 Y. Huang & H. Wu (21.0, ) as the prior for log d (the small variance indicated that we used an informative or strong prior for log d). Similarly, for the following two numerical examples we choose the values of the hyperparameters as follows: a ¼ 4:5, b ¼ 9:0, n ¼ 8:0, L ¼ diag(1000:0, 0:0025, 0:0025, 0:0025, 0:0025, 0:0025, 0:001) h ¼ (0:0,1:1, 1:0, 4:6, 2:3, 6:9, 11:6) T, V ¼ diag(1:25, 2:5, 2:5, 2:0, 10:0, 2:0, 2:0) These values of the hyper-parameters are determined based on several studies published in Han et al. (2002); Ho et al. (1995); Perelson (1989); Perelson et al. (1993, 1996), Perelson & Nelson (1999); Putter et al. (2002); Wei et al. (1995). It can be seen that a non-informative prior is used for log f, while the informative priors are chosen for the other six parameters. Simulated Data Example In this example, we simulate a clinical trial with eight HIV infected patients in a period (16 weeks) of ARV treatment. For each patient, we assume that measurements of viral load are taken at days 0, 7, 14, 28, 56, 84 and 112 of follow-up. The design of this simulation experiment is based on an actual AIDS clinical trial that we will describe in detail latter. For each subject, we generate C min, t r, I 0 and I r in the antiviral drug efficacy model (6) using the uniform distributions with a range of (45.0, 100.0), (30, 150), (0.05, 1.0) and (10.0, 70.0), respectively. These ranges were chosen based on published literature (Molla et al., 1996; Wainberg et al., 1996) and pharmaceutical product information (Abbott Laboratories; GlaxosmithKline; pharmaceutical product information). In addition, a perfect adherence (A(t) ¼ 1 during treatment) is considered for convenient implementations. Thetrueparametersu i ¼ (logf i,logc i,logd i,logl i,logr i,logn i,logk i ) T are generated using the between-subject variation model (9). It is assumed that u i is the linear function of population parameter vector m whose value is m ¼ (0:0, 1:1, 1:0, 4:6, 2:3, 6:9, 11:6) T. We also assume that the random effects b i are normally distributed with mean 0 and standard deviation matrix diag(0:05, 0:01, 0:05, 0:1, 0:05, 0:05, 0:08). Thus, we can obtain the corresponding true values of parameter b i ¼ (f i, c i, d i, l i, r i, N i, k i ) T (i ¼ 1,..., 8) shown in Table 2 later. For the ith patient the baseline viral load, V 0i,is chosen from the actual clinical study introduced in the next subsection. Thus, we can numerically solve the viral dynamic system of differential equations (1) and obtain a numerical solution of viral load for each subject. Model (8) can be used to generate the viral load observations with measurement errors. In our simulations, we assume that the variance of measurement error is s 2 ¼ 1. It is worth noting that the HIV-1 RNA assay has a limit of detection (50 copies/ml, for example) (Wu & Ding, 1999). We simply imputed such values as 50 copies/ml (¼1.7 in log 10 scale). Figure 2(a) displays the viral load trajectory (in log 10 scale) for eight simulated subjects. We then use the generated observations to estimate the parameters by applying the method introduced in the previous section. To examine the dependence of posterior mean on the prior distributions and initial values, we investigated the sensitivity of parameter estimates against prior information and initial values. We follow the method proposed by Raftery & Lewis (1992) to implement the MCMC sampling scheme using FORTRAN codes with calling differential equation subroutine solver (DIVPRK) in IMSL library (1994) and monitor several independent MCMC runs, starting from different initial values. Those runs exhibit similar and stable behaviour. Note that two alternative mean vectors h of prior

12 Estimating Antiviral Efficacy in HIV Dynamic Models 165 Figure 2. Plasma HIV-1 RNA data (log 10 scale). The HIV-1 RNA measurements below a limit of detection of 50 copies/ml are imputed by 50 copies/ml distributions (higher level and lower level, compared with the mean vector h used above) were chosen in sensitivity analyses. The results were more sensitive to the priors than to the initials, but both results are reasonable and robust (data not shown). The conclusions of our analysis remain unchanged. An informal check of convergence based on graphical techniques is used according to the suggestion of Gelfand & Smith (1990). As an example, the number of MCMC iterations and convergence diagnostics with regard to three different initial values are displayed in Figure 3. Based on the results displayed in Figure 3, we propose that, after an initial number of 50,000 burn-in iterations, every fifth MCMC sample is retained from the next 200,000 samples. Thus, we obtain 40,000 samples of targeted posterior distributions of the unknown parameters.

13 166 Y. Huang & H. Wu Figure 3. The number of MCMC iterations and convergence diagnostics with regard to three different initial values based on the simulated dataset. The horizontal long-dash lines denote the true values of parameters Table 1 presents a summary of the results of estimated population parameters. Figure 4 shows both prior distribution densities and posterior densities resulting from the MCMC sampling procedure for all of the parameters. For the drug efficacy parameter log f, which is the parameter of our primary interest, a non-informative prior is given. The posterior mean for f is with standard deviation of , and the 95% credible interval is (0.6913, ), while the true value of f is in our simulation. Thus, the results are very reasonable. For other six nuisance parameters, strong informative priors are given. As we expected, the posterior distributions of these parameters are similar to the priors (Figure 4). In Table 2, we present the estimates of subject-specific individual parameters. The drug efficacy threshold for treatment failure is also given. For comparison, the true parameter

14 Estimating Antiviral Efficacy in HIV Dynamic Models 167 Table 1. The true values (TV), posterior means (PM), standard deviations (SD) and 95% credible intervals (CI) of population parameters based on the simulated dataset Parameter TV PM SD 95% CI f (0.6913, ) c (2.6941, ) d (0.3389, ) l ( , ) r (0.0894, ) N ( , ) k 9: : : (8: ,9: ) Figure 4. Marginal posterior probability density estimates for parameters. log (f) has noninformative prior, while the remaining six parameters have informative priors

15 168 Y. Huang & H. Wu Table 2. The posterior means (PM), true values (TV), drug efficacy thresholds (e) and bias of subject-specific individual parameters based on the simulated dataset Patient f i c i d i l i r i N i k i e PM : TV : Bias : PM : TV : Bias : PM : TV : Bias : PM : TV : Bias : PM : TV : Bias : PM : TV : Bias : PM : TV : Bias : PM : TV : Bias : values and estimation bias are also presented for each individual. It can be seen that for the individual parameter estimates, although the bias ranged differently for different parameters, the bias of the estimate for f i is very small (ranging from to 0.278). The estimates of other nuisance parameters, although strongly depending on the prior, are also reasonable. The fitted curves for eight simulated subjects are displayed in Figure 5. It can be seen that the model seems to provide a good fit. The drug efficacy g (t) is a function of f. After we obtain the estimate ^f of f, we may get the estimate of g (t) as ^g(t) ¼ C min A(t) ^f IC 50 (t) þ C min A(t) Figure 6 gives the estimated g (t) and corresponding true g (t) for the eight simulated subjects. The drug efficacy thresholds e for viral rebound are also given in the plots. Notice that, if g (t) falls below the threshold e, viral load will rebound. Comparing the plots in Figures 5 and 6, g (t) for the simulated patients 1, 2 and 5 is always above the threshold e; thus, their viral load does not rebound. In contrast, for patients 3, 4, 6, 7 and 8, their drug efficacy g (t) falls below the threshold at some time point, thus their viral load starts to rebound at those time points. This fact further confirms the analytic result discussed in Huang et al. (2003) which was briefly summarized in the second section.

16 Estimating Antiviral Efficacy in HIV Dynamic Models 169 Figure 5. Plasma HIV-1 RNA copies (log 10 scale) and fitted curves for eight patients. The circles (W) and the solid lines ( ) denote the simulated viral load and corresponding fitted curves Application to an AIDS Clinical Trial Dataset For the real data example, the data of viral load were taken from an AIDS clinical trial. In the study, 42 HIV infected patients were treated with a combination of anti-retroviral therapy consisting of indinavir (IDV), ritonavir (RTV) and two reverse transcriptase inhibitors (RTIs). Plasma HIV-1 RNA (viral load) measurements were taken at days 0, 7, 14, 28, 56, 84, 112, 140 and 168 of follow-up. The nucleic acid sequence-based amplification (NASBA) assay was used to measure plasma HIV-1 RNA (viral load). Similar to the simulation example, we also considered the viral load data up to 16 weeks (112 days). The viral load trajectories for the 42 patients are shown in Figure 2(b). The data for PK parameters

17 170 Y. Huang & H. Wu Figure 6. The time course of (estimated) antiviral drug efficacy for eight patients. The solid and dotted lines correspond to the true and estimated values based on simulated dataset, respectively; the horizontal lines denote the corresponding thresholds of drug efficacy Table 3. The posterior means (PM), standard deviations (SD) and 95% credible intervals (CI) of population parameters based on an AIDS clinical trial dataset Parameter PM SD 95% CI f (13.69, 50.51) c (2.80, 3.40) d (0.38, 0.41) l (88.14, ) r (0.090, 0.109) N (878.84, ) k 9: : (8:6 10 6,9:810 6 )

18 Estimating Antiviral Efficacy in HIV Dynamic Models 171 Figure 7. The estimated individual parameters and drug efficacy thresholds (e) of subject-specific individuals based on an AIDS clinical trial dataset with 42 patients. SD and CV ¼ SD/Mean denote the standard deviation and coefficient of variation, respectively (C min ), phenotype marker (baseline and failure IC 50 s) and adherence from this study were also used in our modelling. We used the same strategy as the simulation study to implement the MCMC, the same burn-in and sample size were used. We summarize the population estimates in Table 3 and individual estimates in Figure 7. The posterior mean (estimate) for the population parameter f is with a standard

19 172 Y. Huang & H. Wu deviation of 9.07 and the 95% credible interval (13.69, 50.51). As mentioned earlier, f plays a role of transforming the in vitro IC 50 into in vivo IC 50. Our estimate shows that there is a large difference between in vitro IC 50 and in vivo IC 50 in this patient population. The individual-specific parameter estimates, presented in Figure 7, suggest a large between-subject variation since the coefficient of variation (CV) ranges from 40.83% to % for different dynamics parameters. Similar to the simulated example, the model provides a good fit to the clinical data (fitted curves not shown) for 90% of patients. It is also found by comparing the results of fitted curves and estimated drug efficacies that, in general, if ^g (t) falls below the threshold e, viral load rebounds, and in contrast, if ^g(t)is above e, the corresponding viral load does not rebound. This again confirms our theoretical results for the viral dynamic models (Huang et al., 2003). Discussion The main aim of this paper is to investigate a Bayesian approach for estimating parameters, particularly antiviral drug efficacy parameters, in HIV dynamic models during a long-term treatment period. We have developed a modified HIV dynamic model with time-varying antiviral drug efficacy, which is a system of nonlinear differential equations. Bayesian nonlinear mixed-effects models were introduced to characterize the population viral dynamics and to improve estimation efficiency of model parameters. We have presented analyses of two datasets (simulated and real data) to illustrate how the Bayesian method can be applied to studies of HIV dynamics. By concentrating on the estimation of parameters in HIV dynamic models, we have aimed to show that a Bayesian approach is flexible and powerful in fitting complex models such as nonlinear differential equation models. As mentioned, the NLS method is a simple and popular approach to estimate parameters in a nonlinear system, but the parameters for each subject need to be estimated independently, which requires more data from each subject and may not be efficient. The Bayesian approach offers several advantages over the NLS method for estimating parameters in HIV dynamic models. First, a Bayesian approach is a powerful way for incorporating prior information from the existing viral dynamic studies. Not only the point estimates of viral dynamic parameters from previous studies can be incorporated, but also the uncertainty of these estimates can be considered in the analysis. This is very important when the current clinical data are not enough to identify all viral dynamic parameters. Thus, a Bayesian method is a powerful and efficient tool to process all the information available. Secondly, a Bayesian approach provides full posterior distributions of parameters, not just point estimates, which can give a complete inference for the parameters of interest. The results are easy to interpret. Thirdly, the Gibbs sampler or related refinements of MCMC methods under the Bayesian framework provide flexible and powerful tools to estimate parameters in complex models. It is worth noticing that, in combination with the measurements of C min, A(t) and IC 50 (t) from clinical trials and the estimated value of f using the method proposed in this paper, we can estimate time-varying drug efficacy g (t) introduced in (6) during the course of antiviral treatment. As discussed in the previous section, it was interesting to note that, by comparing the results of fitted curves and estimated drug efficacy ^g(t) (obtained in combination with the clinical data C min, A(t), IC 50 (t) and the estimated values of f), if ^g(t) falls below the threshold e, the viral load rebounds, and in contrast, if ^g(t) is above e, the corresponding viral load does not rebound. Thus, the threshold of drug efficacy e may reflect ability of the immune system of a patient for controlling virus replications. It is therefore important to estimate e for each patient based on clinical data. Also notice that our method

20 Estimating Antiviral Efficacy in HIV Dynamic Models 173 can be extended to more complicated models similar to those in (Huang et al., 2003; Nowak & May, 2000; Perelson & Nelson, 1999), although they may contain more parameters to be estimated. However, as proposed in Putter et al. (2002), CD4 data may also be used to help identify more parameters. Thus, parameter identification is an important issue, which needs further attention in future. Acknowledgements The authors are extremely grateful to a referee and the editor for their insightful comments and suggestions that greatly improved the article. This work was supported in part by National Institutes of Health (NIH) research grants RO1 AI and RO1 AI References Abbott Laboratories, Beal, S. L. & Sheiner, L. B. (1985) Methodology of population pharmacokinetics, in: E. R. Garrett & J. L. Hirtz (Eds) Drug Fate and Metabolism-Methods and Techniques (New York: Marcel Dekker). Bennett, J. E. et al. (1996) Markov chain Monte Carlo for nonlinear hierarchical models, in: W. R. Gilks et al. (Eds) Markov Chain Monte Carlo in Practice (London: Chapman & Hall), pp Besch, C. L. (1995) Compliance in clinical trials, AIDS, 9, pp Boeckmann, A. J. et al. (1989) NONMEM Users Guides. Technical report, Division of Clinical Pharmacology, University of California at San Francisco. Bonhoeffer, S. et al. (1997) Evaluating treatment protocols to prevent antibiotic resistance, Proc. Natl. Acad. Sci. USA, 94, pp Carlin, B. P. & Louis, T. A. (1996) Bayes and Empirical Bayes Methods for Data Analysis (London: Chapman & Hall). Chib, S. & Greenberg, E. (1995) Understanding the Metropolis Hastings algorithm, The American Statistician, 49, pp Davidian, M. & Giltinan, D. M. (1995) Nonlinear Models for Repeated Measurement Data (London: Chapman & Hall). Ding, A. A. & Wu, H. (1999) Relationships between antiviral treatment effects and biphasic viral decay rates in modeling HIV Dynamics, Mathematical Biosciences, 160, pp Fitzgerald, A. P. et al. (2002) Modelling HIV viral rebound using non-linear mixed effects models, Statistics in Medicine, 21, pp Galecki, A. T. (1998) NLMEM: a NEW SAS/IML macro for hierarchical nonlinear models, Computer Methods and Programs in Biomedicine, 55, pp Gamerman, D. (1997) Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference (London: Chapman & Hall). Gelfand, A. E. & Smith, A. F. M. (1990) Sampling-based approaches to calculating marginal densities, Journal of the American Statistical Association, 85, pp Gelfand, A. E. et al. (1990) Illustration of Bayesian inference in normal data models using Gibbs sampling, Journal of the American Statistical Association, 85, pp Gelman, A. et al. (1996) Physiological pharmacokinetic analysis using population modeling and informative prior distributions, Journal of the American Statistical Association, 91, pp Geman, S. & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images, IEEE Transactions on Pattern Recognition and Machine Intelligence, 6, pp GlaxoSmithKline, Product Information Han, C. et al. (2002) Bayesian analysis of a population HIV dynamic model, Case Studies in Bayesian Statistics, Vol. 6 (New York: Springer-Verlag). Ho, D. D. et al. (1995) Rapid turnover of plasma virions and CD4 lymphocytes in HIV-1 infection, Nature, 373, pp Hsu, A. et al. (2000) Trough concentratios-ec 50 relationship as a predictor of viral response for ABT-378/ritonavir in treatment-experienced patients. 40th Interscience Conference on Antimicrobial Agents and Chemotherapy, San Francisco, CA; Poster session 171. Huang, Y. et al. (2003) Modeling HIV dynamics and antiviral responses with consideration of time-varying drug exposures, sensitivities and adherence, Mathematical Biosciences, 184, pp