Optimization in Medicine

Transcription

1 Optimization in Medicine INFORMS Healthcare Conference, Rotterdam, 2017 Brian Denton Department of Industrial and Operations Engineering University of Michigan

2 Optimization in Medicine OR in Medicine Cancer Diabetes Kidney Disease Heart Disease

3 Publications on PubMed in the last 10 years: Publications on PubMed in the last 10 years 1,426,842 articles on cancer 169,076 articles on breast cancer 79,662 articles on prostate cancer 474, 417 articles on heart disease and stroke 304,406 articles on diabetes 43,887 articles on kidney disease 2,935 articles on allergies

4 PubMed Results Over the Last 10 Years A Long History ?

5 Ideas to OR in Medicine 1. Optimization can improve medical decision making 2. Medicine can improve optimization 3. There are many unaddressed opportunities for future impact

6 A Long History

7 Example 1: Radiation Treatment External beam radiation is passed through the body harming cancerous and healthy tissue Objective: minimize damage to healthy tissue while delivering required dose to cancer tissue Bahr et al, 1968, The Method of Linear Programming Applied to Radiation Treatment Planning, Radiology, 91,

8 Example: Radiation Treatment Radiation is delivered via a rotating gantry with a multi-leaf collimator Rotating Gantry Multileaf Collimator

9 2-Beam Problem Treatment Brain Cancer: 2-Beam Example Beam 1 Beam 2 1. Tumor 2. Spine 3. Brain Adapted from Optimization in Operations Research, Ron Rardin

10 Linear Program Decision Variables: Exposure times for beams 1 and 2 (x 1, x 2 ) Area Dose Absorbed per millisecond Beam 1 Dose Beam 2 Dose Restriction on Dosage in Kilorads Brain 0.4 x x 2 Minimize Spine 0.3 x x Tumor 0.5 x x 2 6 Center of tumor = 0.6 x x 2 6

11 Linear Program Min σ l L G l (z) Subject to: z j = D kj x k, for all j in V k K x k 0, k K, z j 0, j V z j : the dose delivered to voxel j V x k : the duration of beam k K Romeijn & Dempsey. (2008). Intensity modulated radiation therapy treatment optimization. TOP, 16(2), 215.

12 Extensions Integrated optimization of aperture design and beam intensities: Predefined number of beams Each beam is decomposed into a rectangular grid with m rows and n columns to create an intensity matrix For each row there are 1 n n combinations of left and 2 right leaf settings 1 2 n n m apertures Column generation method: Start with a restricted set of apertures, price out new apertures (columns) via decomposition algorithms Romeijn, E., Ahuja, R., Dempsey, J.F., Kumar, A "A column generation approach to radiation therapy treatment planning using aperture modulation." SIAM Journal on Optimization 15(3);

13 Improvements in Radiation Therapy Path from Research to Implementation Integer programming Convex approximations Inverse Optimization Stochastic and Robust Optimization Vendor Software for Radiologists Linear programming Advances in Mathematical Programming

14 Example 2: Kidney Disease Principal treatment options: Dialysis (home or clinic) Transplant (live or deceased donor) More than 350,000 people are on dialysis and 80,000 waiting for transplant

15 Nonlinear Optimization Miller, J.H. et al Optimization of Certain Parameters in Hemodialysis, Transactions - American Society for Artificial Internal Organs, 6(1); Optimal time to change bath Optimal time to change bath

16 Optimization of Kidney Transplants Kidney Exchange Donors X X Recipients

17 Paired Matching Segev, D, Gentry, S.E., Warren, D.S, Reeb, Montgomery, RA, 2005, Kidney Paired Donation and Optimizing the Use of Live Donor Organs, JAMA, 293(15),

18 Criteria Criteria for Donation Number of matches Number of priority matches Immunologic concordance Travel requirements

19 Example: Constraints Cross-matching Compatibility is determined by two primary factors: Blood type Tissue antibodies Blood type compatibility Donor O A B AB Recipient O A B AB

20 Matching Problems Given a graph G(V, E) a matching is a set of pairwise nonadjacent edges. w 1 w 2 w 3 w 4 w 6 w 5 w 2 w 6 w 1 w 3 w 6 2 matches 3 matches A maximal edge-weight matching is a set of nonadjacent edges with maximum total weight among all matches.

21 Maximum Edge Weight Matching A matching problem for a graph G V, E can be expressed as an integer program Max σ e E w e x e Subject to: σ e v x e 1, for all v V x e {0,1}, for all e E Edmonds, J "Paths, trees, and flowers," Canadian J. Math. 17:

22 Properties of Matching Graphs Analysis of factors that influence vertex and edge weights In a vertex weighted graph with positive weights any matching with maximum vertex weight has maximum cardinality In a matching with maximum edge weight could have half as many edges as a maximum cardinality matching The ratio can be bounded by controlling : max i Connections to multi-criteria problems: Weighted objectives Bi-level optimization w i min i w i Gentry, S., Michael, T.S., Segev, D. Maximum Matching in Graphs for Allocating Kidney Paired Donation, Working Paper

23 Impact Rapid growth of paired donation Year From 1 in 1999, to nearly 600 in 2013, KPD now comprises 10% of living kidney donations ** *Figure courtesy of Sommer Gentry, US Naval Academy;

24 Example 3: Diabetes 29 million people have diabetes in the U.S. 9% of the U.S. population 90% have type 2 diabetes Health complications include micro and macrovascular events Medication can control major risk factors like blood sugar, cholesterol and blood pressure. Mason, J.E. et al Optimizing the Simultaneous Management of Blood pressure and Cholesterol for Type 2 Diabetes Patients. European Journal of Operational Research; 233(3)

25 Sequential Decision Making Choose the best action each time period to maximize long term expected rewards Expected benefit of treatment Expected benefit of treatment Expected benefit of treatment Y Initiate or Delay Treatment? N Change in Health Status Initiate or Delay Treatment? Change in Health Status Initiate or Delay Treatment? Change in Health Status Age 40 Age 41 Age 42

26 State Transition Diagram CVD Events On Treatment Health States before an event has occurred. r(l,w) L r(m,w) M H r(h,w) V r(v,w) Death

27 Markov Decision Process Health status: s t S {1,2,3,.. L, L + 1} Treatment decision in state s t : a s t A(s t ) Optimality Equations for all s t, t = 1,, T 1: Optimal Reward to Go in Health State s t Period t Reward Discounted Expected Future Reward v t s t = max{r s t, a t + λ p s t s t, a t )v t+1 (s t )} a t s t+1 v T s T = r(s T ) Transition probabilities Boundary condition

28 Reward Function Rewards for each state action pair define the objective function for a Markov decision process Reward for living disease free Cost of CVD Events S CHD M r( s, a ) L( s, a ) (1 )( C ( s ) C ( s ) C ( s )) t t t t t t t Medication Cost

29 Life Years to Event (yrs.) Policy Evaluation Men Australian Canadian European U.S. U.S. (ATPIII*) No Treatment Cost ($)

30 Life Years to Event (yrs.) Optimal Policy vs Guidelines MDP Optimal Tradeoff Curve Men Australian Canadian European U.S. U.S. (ATPIII*) Maximum LYs No Treatment Medication Costs ($)

31 Life Years to Event (yrs.) Optimal Policy vs Guidelines MDP Optimal Tradeoff Curve Women Canadian U.S. U.S. (ATPIII*) Australian European Maximum LYs No Treatment Medication Costs ($)

32 Recent Work: Robust MDPs All models are subject to uncertainty in model parameter estimates and model assumptions Transition probabilities are based on statistical estimates from longitudinal data Rewards are based on estimates of mean patient utility, cost, or other performance measures Robust MDPs (RMDPs) attempt to account for this uncertainty Delage, E., Iancu, D Robust Multistage Decision Making. INFORMS Tutorials in Operations Research

33 Robust MDPs Goal of a standard finite horizon MDP is to find π with respect to a fixed Markov chain with TPM, P: π = argmax π Π N 1 E P [ r t s t, π(s t ) + r N s N ] t=1 An RMDP can be viewed as a sequential game against an adversary: π = argmax π Π N 1 min P U EP [ t=1 r t s t, π(s t ) + r N s N ]

34 Time Varying Model This problem is easy when the rectangularity assumption is made: U = s t S U(s t ) Under this assumption the optimality equations are: v t s t = max a t A r t s t, a t + max p s t U(s t ) λ s t+1 S p(s t+1 s t, a t ) v t+1 s t+1 Where p(s t ) is the row of the TPM corresponding to state s t and U(s t ) is the row s uncertainty set.

35 RMDP Case Study: Type 2 Diabetes Many medications that vary in efficacy, side effects and cost. Oral Medications: Metformin Sulfonylurea DPP-4 Inhibitors Injectable Medications: Insulin GLP-1 Agonists Zhang, Y., Steimle, L.N., Denton, B.T., Robust Markov Decision Processes for Medical Treatment Decisions, Working Paper, available at Optimization Online.

36 Treatment Goals HbA1C is an important biomarker for blood sugar control But disagreement exists about the optimal goals of treatment and which medications to use

37 Markov Chain for Type 2 Diabetes HbA1C States

38 Uncertainty Set with Budget p s t+1 s t = pƹ s t+1 s t δ L z L s t+1 + δ U z U s t+1, s t+1 U s t = s t+1 S p s t+1 s t = 1 (z L s t+1 + z U (s t+1 )) Γ(s t+1 ) s t+1 z L s t+1 z U s t+1 = 0, s t+1 0 p s t+1 s t 1, s t+1 Properties: Can be reformulated as a linear program For Γ = S can be solved in O( S )

39 Uncertainty Set A combination of laboratory data and pharmacy claims data was to estimate transition probabilities between deciles p s s, a = n s, s, a σ s n s, s, a, s, s, a 1 α confidence intervals for row s of the TPM: [ pƹ s s, a S( pƹ s s, a L, pƹ s s, a + S( pƹ s s, a L] where S( pƹ s s, a L = 2 pƹ s s, a χ s 1,α/2 S ) 1 pƹ s s, a N(s) 1 2 Gold, Ruth Z. "Tests auxiliary to χ 2 tests in a Markov chain." The Annals of Mathematical Statistics 34, no. 1 (1963):

40 Results Quality adjusted life years to first health complications for women with type 2 diabetes Simulated Worst-case

41 Results Mean QALYs versus variance in QALYs to first event for women with type 2 diabetes

42 Other Work Liver Transplants: Alagoz, Maillart, Schaefer, Roberts, Management Science, 2004 Breast Cancer: Maillart, Ivy, Ransom, Dielhl, Operations Research, 2008 HIV: Shechter, Schaefer, Roberts, Operations Research, 2008 Prostate Cancer: Zhang, Denton, Balasubramanian, Shah, M&SOM 2012 Adherence to Screening: Ayer, Alagoz, Stout, Burnside, Management Science, 2015 Colorectal Cancer: Erenay, Alagoz, Said, M&SOM, 2014

43 Optimization in Medicine: The Future

44 A PubMed Long History Search Results ?

45 Linking Decisions Across Time Optimal Treatment Decisions for Diabetes Research Questions: When and how to screen for diseases? When to use diagnostic tests? When to treat? When to stop? Methods for Sequential Decision Making: Decision Analysis Markov decision processes Partially observable Markov decision processes Multi-stage stochastic programming Reinforcement learning

46 Example Difficult real time optimal control problem Must maintain glucose levels within a defined range Current glucose state difficult to predict Cobelli, C, Renard,E., Kovatchev, B Artificial Pancreas: Past, Present, Future, Diabetes, 60,

47 Resource Constrained Decision Making Optimal Treatment Decisions for Diabetes Research Questions: Coordination across medical silos Prioritizing treatment in resource constrained settings: High value health care Constrained burden on patients Primary- care Public Health Specialtycare Methods: Machine Learning Mathematical programming Sequential decision making Intuitive approximations Optimization Domain Experts Statistics

48 Example Optimizing the coordination of imaging decision for cancer staging Statistical error min n kp z jp j p Subject to: m jp z jp α j p z jp = 1, j p z jp 0,1, j, p Merdan, S., Barnett, C., Denton, B.T., Montie, J.E., Miller, D.C., Data Analytics for Optimal Detection of Metastatic Prostate Cancer Working Paper

49 {Optimization} {Statistics} max { f(x) x C} Data Missing Data Bias Causal Uncertainty Inference and Ambiguity

50 Takeaways Key Points Optimization can improve medical decision making and vice versa but It is underutilized and there are many challenges and unexplored opportunities to address this problem

51 Acknowledgements Christine Barnett, University of Michigan Marina Epelman, University of Michigan Sommer Gentry, US Navel Academy Jennifer Mason, University of Virginia Selin Merdan, University of Michigan Lauren Steimle, University of Michigan Edwin Romeijn, GA Tech Nilay Shah, Mayo Clinic Some of this work was funded in part by grants from the CMMI Division at the National Science Foundation

52 Brian Denton Industrial and Operations Engineering University of Michigan These slides (and pictures!) are on my website: