Free Final-Time Optimal Control for HIV Viral Dynamics

Transcription

1 Free Final-Time Optimal Control or HIV Viral Dynamics arxiv:.3398vmath.oc] Oct Gaurav Pachpute Departmen Mathematics, Indian Institute o Technology Guwahati Guwahati, Assam g.pachpute@iitg.ernet.in Abstract In this paper, we examine a well-established model or HIV wild-type inection. The algorithm or steepest descent method or ixed inal-time is stated and a modiied method or ree inal-time is presented. The irst type o cost unctional considered, seeks to minimize the total time o therapy. An easy implementation or this problem suggests that it can be eective in the early stages o treatment as well as or individual-based studies, due to the hit irst and hit hard nature o optimal control. An LQR based cost unctional is also presented and the solution is ound using steepest descent method. It suggests that the optimal therapy must remain high until the patient shows signs o recovery ater which, the therapy gradually decreases. This is in line with the biomedical philosophy. Solution to a modiied problem which includes a weight or total time is approximated using the modiied algorithm. It shows a considerable drop in the total period. We conclude that, a decreased and optimized therapy period can help us increase eiciency as well as the turnover rate or patient care. I. INTRODUCTION The World health organization (WHO) classiies human immunodeiciency virus (HIV) as a pandemic. HIV progresses into acquired immunodeiciency syndrom (AIDS) which leads to immune system ailure. By 6, AIDS has claimed over million lives and over.6% o the world population is inected with HIV[]. HIV is responsible or the selective depletion o CD4 + cells, also known as helper T cells. As they are essential to immune regulation, such depletion leads to an adverse eecn the immune system unctioning []. Even though HIV is rarely atal by itsel, it increases vulnerability towards inections and malignancies [3]. There is no known cure or HIV/AIDS as the available drug regimens ail to eliminate HIV strains in overall population [4]. The antiretroviral treatments available are mostly administered in drug-cocktail orm, also known as highly active antiretroviral therapy (HAART). Even though HAART has been highly eective in treating HIV/AIDS, it leads to numerous side-eects rom hepatitis, liver ailure, cardiovascular malunction, pancreas damage to nausea, diarrhea and depression [3]. In most patients, the therapy is long-term and thereore, the search or optimal therapy stems rom the notion o maintaining a balance between the disease and drug side-eects to minimize patient suering. Kriti Saxena Electronic and Instrumentation Engineering, BITS-Pliani, Dubai Campus, Dubai, UAE kriti7@gmail.com Several models have been proposed or dierent types o HIV therapies. Many o these models study the host-pathogen reaction or the virus strain HIV- through medical studies or numerical simulations [3]. McLean and Nowak [] address the appearance o AZT-resistant strains and its eects on treatment. Agur [6] and Agur and Cojucaru [7] examine the eects o chemotherapy on uninected cells. Much o the research is also dedicated to acquired inections in an HIV patient [3]. Compartment inections are studied and modeled in [8] and [9]. Microphage inections are addressed in [], whereas, transient viremia is discussed in []. In this paper, we discuss the wild-type inection model given in [3]. This model is a combination o several o the previous models, including, [], [3], and []. This model encapsulates our dierent therapies : protease inhibitors, HIV usion inhibitors, T cell enhancer and HIV reverse transcriptase inhibitor. In this paper, we study the treatment with protease inhibitors. II. HIV MODEL FOR WILD-TYPE INFECTION This paper uses the model given by Stengel [3] or wild-type inectious HIV. The model is given by our coupled-odes and their elements are the HIV particles (x ), uninected T h cells (x ), proviral T h cells ( ) and productively inected T h cells ( ) (all in per mm 3 ). The dierent therapies are protease inhibitors (u ), HIV usion inhibitors (u ), T cell enhancer (u 3 ) and HIV reverse transcriptase inhibitor (u 4 ). The model is given by the ollowing ODEs, = a x a x x ( u )+a 3 a 4 ( u ) = a a x x ( u )( u 4 ) a 6 x +x ( +a 7 x ) + + x (+u 3 ) a 8 3 = a x x ( u )( u 4 ) a 9 a 6 4 = a 9 a 4 () The virus particles (x ) have a death rate o a. x inect the uninected T cells (x ) at rate a. This is partially blocked by HIV usion inhibitors with eicacy u. A portion (-u ) o the production o x rom the productively inected cells

2 ( ) is blocked by the protease inhibitors with eicacy u. Birth rate o x is proportional to a and depends on the amoun virus present in the system. The conversion o x to proviral T h cells ( ) occurs at a rate a and is targeted by HIV reverse transcriptase inhibitor (u 4 ). x has a natural death rate o a 6 and a prolieration rate o a 7. Prolieration is enhanced by T cell enhancer (u 3 ). Proviral T h cells ( ) are produced through inection. convert into at a rate a 9 and have a death rate o a 6. The inected cells ( ) convert into virus (x ) at rate a 4 with the ratio :a 3, completing the cycle. The parameter values (rom [3] and []) are a =.4, a =.4, a 3 =, a 4 =.4, a =, a 6 =., a 7 =.3, a 8 =, a 9 =.3. We deine the vectors x = [x x ] and u = [u u u 3 u 4 ], then equation () can be written as ẋ = (x(t),u(t),t). The initial conditions are given by x() = x. We assume that, the drug eicacy or drugi, u i varies between [u i,u i ] and as the eicacies are normalized this is a subset o [,]. In this paper, we discuss the therapy with protease inhibitors (u ) and take u = u 3 = u 4 =. III. OPTIMAL CONTROL THEORY A general optimal control problem can be stated as a combination o a system o ordinary dierantial equations (ODEs), which deines the dynamics o the state variables and controls along with a cost unctional, which is to be minimized. The problem is deined over a time period [,t ], where t can be predeined or ree. Consider a system given by ẋ = (x,u,t), x() = x and a cost unctional given by, J = K(x(t ),t )+ L(x(t),u(t),t)dt. I the inal time is ree, another constraint can be pun the inal state o the system : Ψ(x(t ),t ) =, where Ψ is a smooth unction. Hamiltonian or this system is deined by, H(x,u,λ,t) = L(x,u,t)+λ(t)(x,u,t). The necessary conditions or a minimum are achieved by deining the modiied cost unctional J using lagrangian multipliers λ(t) and p as, J = K(x(t ),t )+pψ(x(t ),t )+ [ L(x(t),u(t),t) λ (t)((x(t),u(t),t) ẋ) ] dt. The cost unctional reaches minimum when the variation, = with respect to all variables, λ,p,x,u and x(t ) (also, t, i inal-time is ree). Using this argument, i x (t) and u (t) are the optimal state and control vectors or time t respectively then we can state the Pontryagin s necessary conditions as ollowing, ) Ψ(x (t ),t ) = (Final State Constraint) ) ẋ = λ H = (x (t),u (t),t) (State Equation) 3) λ = x H (Costate Equation) 4) u H = (Optimal Control) ) x( ) = x λ(t ) = x(t )(K+p Ψ) H(t ) = t (K+p Ψ) (Boundary Conditions) As or the suicient conditions, where one is concerned with the local minimality, it is given by u H > (Positive deinite). For global minimality, one must consider all possible controls over time [,t ]. A Hamiltonian convex in u can ensure the global minimality. I the inal time is ree, only the irst two boundary conditions are valid. Furthermore, i there is no inal state constraint, the second boundary condition becomes, λ(t ) = x(t )K. One o the most widely used cost unctional is the Linear- Quadratic Regulator (LQR) given by, J = x (t )Sx(t )+ x (t)qx(t)+u (t)ru(t) ] dt. The matrices, S, Q, R are the weights or the inal state coniguration, integral value o state and control, respectively. I the inal time is ree, one may add a scalar T in the integral to include the weight or total time. A. Algorithm or ixed inal-time optimal control problems As it can be seen rom the necessary conditions, the optimal control system comprises a orward state equation and a backward costate equation. For nonlinear models the algorithms are essentially iterative. One o the very eective o these algorithms is the steepest descent method. The solutions in this paper are approximated using this algorithm [4]. ) Approximation starts with an initial guess u(t) = u () (t). ) x () (t) is integrated numerically over time [,t ] using control u () (t) with x() = x as the boundary condition. 3) Similarly, the costate equation λ = x ()H is numerically solved with λ(t ) = x(t )K as the boundary condition. 4) The gradient u ()H is evaluated using numerical techniques and the updated control or the next iteration is given by, u () (t) = u () (t) τ() u ()H. Similarly, the k+-th iteration in terms o u k (t) is given by, u (k+) (t) = u (k) (t) τ(k) u (k)h. ) This iterative procedure is repeated until some convergence criterion such as, u (k+) u (k) < tol or J (k) J (k+) < tol is satisied. B. Modiied algorithm or ree-inal time problems I the inal time is ree, the algorithm can be modiied to encapsulate it by updating the control as well as the grid size or numerical integration in every iteration. For a small variation t, one can write J = t. For LQR, this can be written as,

3 J = [ x (t )S t x(t ) ] t + x (t )Qx(t )+u (t )Ru(t )+T ] t = x (t )S(x(t ),u(t ),t ) t + x (t )Qx(t )+u (t )Ru(t )+T ] t () Suppose, one expects to reduce J by %, then one can change t using the ollowing equation, t = t (k+) t (k) =., where, t (k) represents the k-th iteration. Note that, i the number o grid points is n and the grid length at the k-th iteration is h (k) then t (k) = (n )h (k). Thereore, at every iteration the grid length can be updated as, h (k+) = h (k) τ h(k) (n ). This condition corresponds to the third boundary condition speciied in the last section, i.e., = H(t ) + t K. However, this algorithm cannot be eective when the problem is speciied with a inal state constraint (Φ). A. Minimum-time problem IV. RESULTS AND DISCUSSIONS In minimum-time problems, the cost unctional is deined as J = t = t dt. They are oten presented with an acceptable set Ω o inal state variables, i.e., the value o Φ over this set is zero. The problem speciied in the next paragraph can be solved using the iterative method described in the last section by continuing the process until the system either enters the set Ω or leaves it, but as there is no incentive or the control to be any less than its maximum value, another way to deine these problems is to ind the minimum time it takes or the system to enter set Ω. Using this notion, we can solve the problem with an error o the order o the grid size used or numerical integration. For the system at hand, the set Ω is taken as the values o virus particles below a certain level, oten called the detection level. We take the detection level to be. []. The minimum-time problem can be stated as : To ind an admissible control u : [,t ] [u,u ] and the inal time t, which or the system ẋ = (x(t),u(t),t),x() = x minimizes the cost unctional J = t = t such that the inal virion population is below the detection level. Here, the boundary conditions are taken as x () = 3.,x () = 94, () = 3.4, () =.46 and u =.9, [6] or pharmacokinetic models or HIV suggest that the eicacy decays exponentially and a perect eicacy throughout the treatment period is unlikely to be achieved. The solution to this problem is presented in Figure and Figure. The results Uninected T cells Inectious HIV Inected T cells Fig.. States or min-time problem Fig.. Control or min-time problem suggest that it takes about 9 weeks or the virus to climb down below the detection level. This type o method can be used or inding the optimal therapy or curable or highly manageable inections or or drugs with minimal side-eects, where a ull-blast treatment is not detrimental, since the main cause o suering originates rom the time the inection is active in the system. This method can also be useul in an early detection or treatment or the disease, as we shall see in the next subsection that the therapy remains at its highest under these conditions. The maximum eicacy can be averaged over one period o administering drugs or a ull-blast therapy. The easy implementation o the aorementioned procedure leaves room or studies based on individual patient scenarios [7]. B. Fixed inal-time problem In this section, as a prerequisite to the ree inal-time problem, we develop a ixed inal-time cost unctional (ater x x u

4 Uninected T cells Inected T cells Inectious HIV x x Uninected T cells Inected T cells Inectious HIV x x Fig. 3. States or ixed inal-time problem Fig.. States or ree inal-time problem u u Fig. 4. Control or ixed inal-time problem Fig. 6. Control or ree inal-time problem [3]) which sheds lighn some o the key eatures o the optimal control. We take x () = 4.9,x () = 94, () =.34, () =.4 and t = days. The cost unctional is given by, J = x (t )Sx(t )+ [ x (t)qx(t)+u (t)ru(t) ] dt, where, S, Q and R are diagonal matrices. We wish to design a problem that minimized the terminal and integral values o proviral T cells ( ), productively inected cells ( ) and the virus (x ) as well as the integral value o the control (u ). Keeping these goals in mind, we take the weights to be, S = S 33 = S 44 = 3, Q = Q 33 = Q 44 = 3 and R =.3]. The solution is approximated by steepest decent method described in the previous section and is presented in Figure 3 and Figure 4. The results suggest that the optimal therapy (u ) stays at its maximum or about 3 days and then gradually declines over the res the period. This is in sync with the biomedical philosophy o hit irst and hit hard [3], i.e., to treat the disease with high dosage in the early phase o the treatment and once a physical recovery takes place, gradually decrease the dosage. The physical recovery is represented by the uninected T h cell count, which rises to almost its highest in the irst 3 days [4]. Similarly, the proviral count and the productively inected T cells have gone down substantially. Simulations run over a larger or smaller periods along with varied values o S,Q, and R suggest that the period o high eicacy does not vary by much as long as the values o R stay relatively low. This suggests that when treating with drugs with minimal side eects, the length o this period stays approximately the same [4]. This kind o studies o optimal therapy are very common and successul, as they relect the well-accepted conceptions

5 o the bio-medical ield. They do, however, ail to address the time window or which the therapy continues. This is not relected through the drug side-eects, especially, i they are considerably high, or the optimal eicacy changes with the period o treatment. Thereore, it is important to take time into account when trying to minimize the cost unctional. C. Free inal-time problem In this section, we present the model or a ree inal-time LQR. We state the optimal control problem as the ollowing, To ind an optimal control u : [,t ] [,] which, or the system ẋ = (x(t), u(t), t), minimizes the cost unctional, J = x (t )Sx(t )+Tt + x (t)qx(t)+u (t)ru(t) ] dt, where, the weights S, Q and R serve the same purpose as beore and the added term Tt adds a cost or the duration o therapy. The values used are, S = S 33 = S 44 = 3, Q = Q 33 = Q 44 = 3, R =. and T =.. The initial state o the system is taken to be the same as or the previous problem, i.e., x () = 4.9,x () = 94, () =.34, () =.4. To solve this problem, the algorithm described in section is implemented. is given by equation and the grid length h is updated ater every ew iterations. As mentioned in section, this method tries to satisy one o the boundary conditions, namely, H(t )+ t K =. An implementation in MatLab TM shows a considerable drop in this value rom the initial state. The results are presented in Figure and Figure 6. As the results suggest, we notice a substantial drop in the treatment window rom days to about 7 days. However, the essential eatures o the therapy described in the last subsection are retained in Figure 6. The optimal therapy stays at its highest or the irst days and then shows a gradual decrease. This, as noted beore, coincides with the restored T cell count and the decreased adverse element counts in the system. As given in the last subsection, this period o high eicacy varies (rom 3 days to days) when treated over a dierent time window. The state variables show similar progression over time under the two optimal therapies. The method decreases the total period o therapy while keeping the state variables as intact as possible. CONCLUSION In this paper, we consider a well established model or wild-type HIV inection. A common LQR optimal therapy is approximated with the steepest descent method. We, also, present a modiied algorithm or approximating the solution or ree inal-time problems. A minimum-time problem is stated and the solution is approximated with a non-iterative method. We conclude that, due to the simple implementation and the hit irst and hit hard nature o the optimal therapy, this kind o studies can be very useul in the irst phase o therapy and also provide an advantage or individual-based optimal-therapy considerations. The second cost unctional is an LQR deined over a ixed time period, which indicates similarities o the optimal control with biomedical philosophies. The optimal therapy shows a period o high eicacy in the irst phase and a gradual decrease once there is physical progress in patient. A modiied problem with ree inal-time is approximated using the algorithm mentioned earlier. It shows a considerable drop in the total period. An optimized time window or treatment can help us treat patients more eiciently. More number o patients can be treated i the optimal period or therapy is lower. The method presented in this paper is quite similar to methods already in use. The need or optimizing the therapy period is quite essential as a higher or lower time period suggests a non-optimal exposure to either the disease or therapy. REFERENCES [] Overview o the global AIDS epidemic (6), Reporn the global AIDS epidemic. [] A.S. Perelson (993), D.E. Kirschner, R.J. DeBoer, Dynamics o HIV inection o CD4+ T cells, Mathematical Biosciences, 4: 8. [3] Stengel RF (8), Mutation and control o the human immunodeiciency virus, Mathematical Biosciences, 3(): 93- [4] M. DiMascio, R.M. Ribeiro, M. Markowitz, D.D. Ho, A.S. Perelson (4), Modeling the long-term control o viremia in HIV- inected patients treated with antiretroviral therapy, Math. Biosci. 88: 47. [] McLean A, Nowak M (99), Competition between AZT sensitive and AZT resistant strains o HIV. AIDS 6, 779 [6] Agur Z (989), A new method or reducing cytotoxicity o the anti-aids drug AZT. In: D. S. Levine, J. C. Baltzer: AG, Biomedical Modeling and Simulation, (IMACS), pp. 96, Scientic Publishing Co [7] Cojocaru L. and Agur Z. (993): New mathematical methods or evaluating toxicity o anti-cancer drugs and AZT. [8] A.S. Perelson, P. Essunger, Y. Cao, M. Vesanen, A. Hurley, K. Saksela, M. Markowitz, D.D. Ho, Decay characteristics o HIV--inected compartments during combination therapy, Nature 387 (997) 88. [9] S.H. Bajaria, G. Webb, M. Cloyd, D. Kirschner, Dynamics o nave and memory CD4+ T lymphocytes in HIV- disease progression, J. Acquir. Immune Deic. Syndr. 3 () () 4. [] D. Wodarz, A.L. Lloyd, V.A.A. Jansen, M.A. Nowak, Dynamics o macrophage and T cell inection by HIV, J. Theor. Biol. 96 (999). [] L.E. Jones, A.S. Perelson, Opportunistic inection as a cause o transient viremia in chronically inected HIV patients under treatment with HAART, Bull. Math. Biol. 67 () 7. [] Kirschner D, Lenhart S, Serbin S (997), Optimal control o the chemotherapy o HIV, Journal o Mathematical Biology 3: 77. [3] Perelson AS (989), Modeling the interaction o the immune system with HIV, in: C. Castillo-Chavez (Ed.), Mathematical and Statistical Approaches to AIDS Epidemiology, Springer-Verlag, New York, p.. [4] Pachpute G, Chakrabarty SP (), Optimal Therapy o Hepatitis C Dynamics and Sampling Based Analysis, [] Janet M. Barletta, Daniel C. Edelman, and Niel T. Constantine (4), Lowering the Detection Limits o HIV- Viral Load Using Real-Time Immuno-PCR or HIV- p4 Antigen, Microbiology and Inectious Deseases. :-7 [6] Anita J. Brogan, Joseph Mrus, Andrew Hill, Anthony W. Sawyer, and Erik Smets (), Comparative Cost-Eicacy Analysis o Darunavir/ritonavir and Other Ritonavir-Boosted s or First-Line Treatmen HIV- Inection in the United State, HIV clinical trials, /3 : p [7] Pachpute G, Chakrabarty SP (), Analysis o Hepatitis C Viral Dynamics Using Latin Hypercube Sampling,