OR Forum A POMDP Approach to Personalize Mammography Screening Decisions

Transcription

1 OPERATIONS RESEARCH Vol. 60, No. 5, Sepember Ocober 2012, pp ISSN X prin) ISSN online) hp://dx.doi.org/ /opre INFORMS OR Forum A POMDP Approach o Personalize Mammography Screening Decisions Turgay Ayer H. Milon Sewar School of Indusrial and Sysems Engineering, Georgia Insiue of Technology, Alana, Georgia 30332, ayer@isye.gaech.edu Oguzhan Alagoz Deparmen of Indusrial and Sysems Engineering, Universiy of Wisconsin, Madison, Madison, Wisconsin 53706, alagoz@engr.wisc.edu Naasha K. Sou Deparmen of Populaion Medicine, Harvard Medical School/Harvard Pilgrim Healh Care Insiue, Boson, Massachuses 02115, naasha_sou@hms.harvard.edu Breas cancer is he mos common nonskin cancer and he second leading cause of cancer deah in U.S. women. Alhough mammography is he mos effecive modaliy for breas cancer screening, i has several poenial risks, including high falseposiive raes. Therefore, he balance of benefis and risks, which depend on personal characerisics, is criical in designing a mammography screening schedule. In conras o prior research and exising guidelines ha consider populaion-based screening recommendaions, we propose a personalized mammography screening policy based on he prior screening hisory and personal risk characerisics of women. We formulae a finie-horizon, parially observable Markov decision process POMDP) model for his problem. Our POMDP model incorporaes wo mehods of deecion self or screen), age-specific unobservable disease progression, and age-specific mammography es characerisics. We solve his POMDP opimally afer seing ransiion probabiliies o values esimaed from a validaed microsimulaion model. Addiional published daa is used o specify oher model inpus such as sensiiviy and specificiy of es resuls. Our resuls show ha our proposed personalized screening schedules ouperform he exising guidelines wih respec o he oal expeced qualiy-adjused life years, while significanly decreasing he number of mammograms and false-posiives. We also repor he lifeime risk of developing undeeced invasive cancer associaed wih each screening scenario. Subjec classificaions: parially observable Markov decision processes; dynamic programming; decision analysis; medical decision making; breas cancer; mammography screening; personalized screening. Area of review: Decision Analysis. Hisory: Received March 2010; revisions received July 2010, Ocober 2010, December 2010, February 2011, March 2011; acceped June Published online in Aricles in Advance June 12, Inroducion For years, mammograms have been recommended every year or wo for women beginning a age 40. The new repor from he US Prevenive Services Task Force, issued Monday nigh, now says women his age should simply alk o heir docors abou he benefis and risks. USA Today 2009). Breas cancer is he mos common cancer among U.S. women, resuling in more los years of poenial life han any oher cancer because of is higher occurrence in younger ages han mos cancers American Cancer Sociey ACS) 2009). Abou one in eigh U.S. women will develop breas cancer in heir lifeime. In 2009 alone, an esimaed 254,650 women were diagnosed wih breas cancer and 40,170 died from his disease, making i he second leading cause of cancer deahs in U.S. women ACS 2009). Alhough here is no guaraneed way o preven breas cancer, when deeced early he disease is more likely o be curable. For example, he five-year survival rae increases from 27% o 98% when breas cancer is deeced a early sages compared wih laer sages ACS 2009). Mammography is he mos commonly used, and he only proven, screening modaliy o deec preclinical breas cancers. On average, mammography deecs cancer 1.7 years before a woman can feel he lump and several years before physical sympoms develop Ceners for Disease Conrol and Prevenion CDC) 1995). The evidence suggess ha mammography has a poenial o reduce breas cancer moraliy raes by 20% 30% Kerlikowske e al. 1995). Being fas, reasonably accurae, and widely available make mammography he gold sandard for screening. Neverheless, mammography is no perfec and has several poenial risks, including radiaion exposure, pain, overdiagnosis finding cases ha would no have clinically 1019

2 Ayer, Alagoz, and Sou: Personalized Mammography Screening 1020 Operaions Research 605), pp , 2012 INFORMS surfaced and caused breas cancer moraliy in a woman s lifeime), and risks associaed wih false-posiive mammograms, i.e., mammograms wih posiive oucome when he disease is absen Brewer e al. 2007). In paricular, false-posiive mammograms are serious and harmful because hey may lead o unnecessary diagnosic follow-up e.g., addiional imaging and invasive procedures such as biopsy), which may in urn resul in associaed morbidiies, psychological disress including anxiey and depression, considerable amoun of ime loss, and, as a resul, a significan reducion in qualiy of life Brewer e al. 2007). Ye, false-posiive mammograms are no rare 10.7% per mammogram), and he risk increases wih increased screening frequency. For every 1,000 healhy women who undergo annual mammograms, more han half will have a false posiive, and nearly 200 of hem will undergo an unnecessary biopsy wihin 10 years Elmore e al. 1998). The balance of hese healh benefis and poenial risks is herefore criical for designing an effecive mammography screening program. The balance may shif based on cerain personal risk characerisics such as age, family hisory, and pariy Armsrong e al. 2007). For example, he benefis and risks of mammography screening differ in younger versus older women, leading o conroversy among researchers. Some recommend more-frequen screening for younger women and less-frequen screening for older women because breas cancer is hough o be more aggressive in younger women Jayasinghe e al. 2005). In addiion, older women are more likely o suffer from risky comorbidiies oher healh problems increasing he moraliy risk such as sroke, hyperension, and myocardial infarcion), whereas younger women mosly do no have such risks Saariano and Silliman 2003). On he oher hand, here are also reasons o believe ha more-frequen screening is beneficial for older women. For example, he incidence and moraliy of breas cancer increase dramaically wih age ACS 2009). Approximaely 50% of all newly diagnosed breas cancers occur in women aged 65 years and older Holmes and Muss 2003). Furhermore, he accuracy of mammography screening is higher in older women, primarily due o differences in breas densiies among older and younger women Kerlikowske e al. 2000). These conroversies are also refleced in screening guidelines by several major healh organizaions in he Unied Saes and oher counries. Significan variaions exis among differen screening guidelines in erms of he recommended age o sar and end mammography screening, as well as he frequency of screening Table 1). Recenly, he U.S. Prevenive Services Task Force USPTF), a governmen-appoined exper panel whose sance influences coverage of screening ess by Medicare and many insurance companies, has updaed is recommendaions from annual o biennial screening beween ages USPSTF 2009). Several advocacy groups welcomed he new guidelines, whereas ohers, including he ACS, disagreed wih i New York Times 2009). None of hese general populaion-based guidelines consider risk facors oher han age. However, evidence suggess ha oher personal risk facors may also be imporan for balancing he benefis and risks of a cancer-screening sraegy Armsrong e al. 2007). As noed by Gail and Rimer 1998), he ideal screening recommendaion would be one ha refleced each woman s individual risks, which is influenced by several oher personal characerisics such Table 1. Recommended mammography screening policies by various insiuions in he Unied Saes Qaseem e al. 2007, USPSTF 2009) and oher counries wih organized populaion-based cancer-screening programmes Klabunde and Ballard-Barbash 2007, Shapiro e al. 1998). Screening inervals in years Insiuion/Counry Sar age End age Age Age 50 + American Cancer Sociey, 40 No specified 1 1 American Medical Associaion, American College of Radiology Naional Cancer Insiue 40 No specified U.S. Prevenive Services NA 2 Task Force American College of Prevenive 50 No specified NA 1 2 Medicine American Academy of Family 50 No specified NA 1 2 Physicians American College of Obsericians and 40 No specified Gynecologiss Canada, Ialy, Japan NA 2 France, Neherlands NA 2 Spain Sweden Unied Kingdom NA 3 In Sweden, he 1.5-year inerval is applied o age group 40 54; he 2-year inerval is applied o age group NA: No applicable.

3 Ayer, Alagoz, and Sou: Personalized Mammography Screening Operaions Research 605), pp , 2012 INFORMS 1021 as family hisory, breas densiy, body mass index BMI), alcohol consumpion, pariy, exen of breasfeeding, and ages a menarche, menopause, and firs birh. For example, a woman wih a family hisory of breas cancer is wo o hree imes more likely o develop breas cancer compared wih a woman who has no family hisory Gilbar 1998). Undersanding ha women a he same age groups do no have uniform breas cancer risks suggess ha screening sraegies ha are ailored o individual risks may be more beneficial in increasing life-savings in high-risk women while decreasing unnecessary complicaions in low-risk women IOM 2005). In fac, personalized mammographyscreening sraegies are idenified in he 2005 Insiue of Medicine IOM) repor as crucial o improving he early deecion of breas cancer IOM 2005). Despie he growing need for ailored screening, a review of he lieraure indicaes ha no analyical framework ye exiss o plan for personalized screening. The purpose of his sudy is o begin o address his need. We develop a modeling framework o deermine an opimal personalized mammography-screening sraegy ha accouns no only for saic risk facors such as race, age a menarche, and age a firs live birh, bu also for dynamic risk facors including age and prior screening hisory Oher dynamic risk facors such as BMI and a changing family hisory can only be approximaely accouned for by redoing he model compuaions when he change occurs, as we discuss in our concluding secion). We model his problem using a parially observable Markov decision process POMDP), which is a generalizaion of a Markov decision process ha allows sequenial decision making when he informaion regarding he rue sae of he sysem is incomplee. POMDPs allow capuring parial observabiliy of he disease progression and imperfec es resuls, making hem ideally suied o healh-care problems. However, he applicaion of POMDPs o medical decision making has been limied o only a few sudies, primarily due o heir high compuaional complexiy Schaefer e al. 2004). To he bes of our knowledge, his is he firs analyical sudy ha considers how o personalize mammographyscreening policies. The only sudy we are aware of ha suggess risk-based recommendaions for mammography screening is by Gail and Rimer 1998), which proposes o screen for women beween age only if he woman s risk a his age equals or exceeds ha of he one a 50-years old who has no risk facors for breas cancer. For women over age 50, Gail and Rimer assume ha annual screening is opimal. Several oher sudies invesigae populaionbased breas cancer screening policies using simulaion and analyical models see Alagoz e al for an exensive review). Among hese populaion-based analyses, he mos relevan analyical sudy o ours is by Maillar e al. 2008), which builds a parially observable Markov chain and provides an upper bound on lifeime breas cancer moraliy risk by evaluaing numerous alernaive screening scenarios. Anoher relevan sudy is by Ivy 2002), which uses a POMDP srucure for finding a cos-effecive mehod for mammography screening wih respec o he compeing objecives of payers and paiens. She approximaely solves his POMDP assuming ha deah probabiliies and es accuracies are age independen. Our conribuions in his research are wofold: From he applicaion viewpoin, 1) raher han a populaionbased policy, we propose a personalized opimal screening sraegy ha considers several personal risk characerisics of women, including prior screening hisory; 2) we uilize a validaed microsimulaion model, which is used also o develop acual policy recommendaions, o specify our ransiion probabiliy parameers; 3) unlike mos of he analyical models, we incorporae he possibiliy of self-deecion ino he mammography-screening problem; and 4) we acknowledge ha disease progression, moraliy raes, and es accuracies are age dependen. From he heory viewpoin, 1) we build a POMDP model ha is differen han convenional POMDP models and beer suied o many medical decision-making problems; 2) conrary o prior POMDP applicaions for real-life problems in he lieraure, we solve our POMDP opimally; and 3) we derive several srucural properies of our POMDP model, which is rarely done in he lieraure. The assumpions needed o invoke hese resuls a a paricular ime sep depend on he compued parameers from a prior ime sep. Therefore, i is no possible o use hese resuls o predic a conrollimi policy prior o beginning compuaion. Neverheless, we feel ha hese resuls are sill imporan as hey provide useful insighs abou he overall problem. The remainder of his paper is organized as follows. In 2, we presen he POMDP model for his problem. In 3, we describe he model inpus and parameer esimaions. In 4, we presen and discuss compuaional resuls. Finally, we summarize our findings and conclude in 5. An elecronic companion o his paper is available as par of he online version ha can be found a hp://-dx.doi.org/ /opre Model Formulaion We formulae a discree-ime, finie-horizon POMDP model o solve his problem, in which a single decision maker such as a paien and/or a physician aims o maximize he oal expeced qualiy-adjused life years QALYs) of he paien. We assume ha he decision maker is risk neural. Like mos of he exising breas cancer screening guidelines USPSTF 2009), we do no consider he financial coss associaed wih breas cancer screening in his sudy because we ake he paien s perspecive, and hese coss are covered by privae insurance plans and Medicaid in almos all saes Rahore e al. 2000). A he beginning of every six monhs, a woman eiher undergoes a mammogram or is recommended o wai for

4 Ayer, Alagoz, and Sou: Personalized Mammography Screening 1022 Operaions Research 605), pp , 2012 INFORMS anoher six monhs. This decision is made based on he woman s curren risk of breas cancer, which may depend on several personal risk facors as well as prior screening hisory and can be esimaed using one of he mehods explained in 3.3. If he woman is recommended o have a mammogram and if he mammogram urns ou o be posiive, i is followed by a perfec follow-up es e.g., biopsy). This is a reasonable assumpion because he lieraure repors ha biopsy is a reliable procedure wih rue posiive raes very close o 1 Parker e al. 1994). If he resul of he biopsy is also posiive i.e., he woman has cancer), we assume ha he woman sars reamen and quis he decision process by moving o a poscancer sae wih probabiliy 1. This assumpion is in line wih he exising lieraure, because he curren screening guidelines sugges ha women wih a personal hisory of breas cancer have differen biologies; herefore, hey should no follow he general screening guidelines, bu insead hey should be screened more aggressively ACS 2009). On he oher hand, he woman coninues he decision process afer a negaive mammogram, a negaive biopsy following a posiive mammogram, or a recommended Wai acion. If he woman is recommended o wai for six monhs, breas cancer may be deeced a any ime during his period hrough selfdeecion, i.e., eiher hrough clinical breas exam CBE) or breas self-exam BSE) Figure 2). Throughou he paper, we refer o he woman as he paien, irrespecive of her healh condiion, for consisency. The noaion used in he model is as follows. : Decision epochs, = T T <. We assume ha screening decisions are made every six monhs and define as he number of half years above he age 40 e.g., = 0 represens age 40, = 1 represens age 40.5, ec.). The decision epochs sar a age 40 because his is he earlies age among he recommended saring ages for rouine mammography screening Qaseem e al. 2007). We end our decision horizon a age 100 i.e., T = 120), consisen wih he U.S. life ables repored by he CDC Arias 2006). S: Core sae space, S = , where s S represens he rue healh sae of he paien a ime. In paricular, 0 represens a cancer-free paien, 1 represens a paien wih in siu noninvasive) cancer, 2 represens a paien wih invasive cancer, 3 represens a paien under in siu cancer reamen, 4 represens a paien under invasive cancer reamen, and 5 represens deah. Noe ha he decision maker direcly observes wheher he paien is in sae 3, or in sae 4, or in sae 5. However, he decision maker only observes he paien is in one of 0 1 2, bu no which one Figure 1). We separae in siu and invasive cancers in our analysis because risk facors and managemen for hese cancers differ; hus, he benefi of mammography may be differen for each cancer ype Reinier e al. 2007). S : Informaion space, he space of all probabiliy disribuions over he sae space S. Any elemen of S is referred o as an informaion sae, denoed by, which consiss of occupaion probabiliies over he sae space. Figure 1. No cancer 0) Sae ransiion diagram of he underlying Markov process. Deah 5) In siu cancer 1) In siu poscancer 3) Invasive cancer 2) Invasive poscancer 4) Tha is, if we le s denoe he probabiliy of occupying sae s, hen = s. B S PO : Belief space. Any elemen of B S PO is referred o as a belief sae, denoed by b, which is a runcaed version of over he parially observable saes. For example, if = , hen b = Inuiively, b represens he belief of he decision maker abou he paien s parially observable healh saes and is known o be a sufficien saisic for he enire hisory of he process Asrom 1965). a : Acion aken a ime, i.e., a A = W M, where W and M represen Wai and Mammography, respecively. a : Observaion space, which includes observaions seen upon aking acion a. If a = M, he decision maker can observe a posiive mammogram M+) or a negaive mammogram M ). If a = W, he paien can make a self-deecion SD+) or no self-deecion SD ). Tha is, M = M+ M and W = SD+ SD. K a o s : Observaion probabiliy, which represens he probabiliy of making an observaion o a ime when he acion chosen is a and he rue healh sae is s. Wihou loss of generaliy, we define he orderings beween he observaions as follows: SD SD+ and M M+. Observaion probabiliies are specified by he accuracy of he ess, and because hese ess are imperfec, hey provide only parial informaion abou he rue healh sae of he paien. In he medical lieraure, he accuracy of a es is ypically measured by specificiy he proporion of cancer-free women who are idenified as negaive by he es and sensiiviy he proporion of women wih cancer who are idenified as posiive by he es. Le spec M and spec SD denoe he specificiy of mammography and self-deecion a ime, respecively. Similarly, le sens s M and sens s SD denoe he sensiiviy of mammography and self-deecion for rue healh

5 Ayer, Alagoz, and Sou: Personalized Mammography Screening Operaions Research 605), pp , 2012 INFORMS 1023 Figure 2. Mammography decision-making diagram BCa represens breas cancer ). Time Time + 1 Invasive Treamen for invasive BCa Posiive M+) Biopsy In siu Treamen for in siu BCa Mammogram Negaive b + 1 = P M,M+. s = 0) Mammogram M) Deah Wai Mammogram b Negaive M ) b + 1 = [b, a = M, o = M ] Wai Deah Mammogram Wai W ) Self-deecion SD+) No self-deecion SD ) Deah b + 1 = [b, a = W, o = SD+] Wai Mammogram b + 1 = [b, a = W, o = SD ] Wai sae s a ime, respecively. We define sensiiviy as a funcion of he rue healh sae s because, unlike specificiy, sensiiviy depends on he cancer sage i.e., in siu or invasive). Then, he observaion probabiliies a ime are compued as follows: K M K M K W M s = 0 = spec M M+ s = 0 = 1 spec M SD s = 0 = spec SD K W SD+ s = 0 = 1 spec SD and K M K M K W M+ s = sens s M M s = 1 sens s M SD+ s = sens s SD K W SD s = 1 sens s SD for s 1 2 a o P s s : Core sae ransiion probabiliy, i.e., he probabiliy ha he paien will be in sae s S a ime + 1, given ha she was in sae s, ook acion a, and observed o a ime. The decision process is presened in Figure 2. The probabiliy of a change in healh sae in he subsequen inerval is he same for women who have a negaive or a false-posiive mammogram and hose who do no M M receive a mammogram Figure 2). Tha is, P s s W SD = P s W SD+ s = P s s for all s s S and M M+ M M P s 0 = P s 0. Because informaion updaes occur a discree imes, we updae he corresponding informaion sae a he end of his decision epoch. Noe ha a SD+ may change he opimal acion aken in he fuure by increasing he belief abou he cancer risk. b a o : Updaed belief sae, which capures he informaion on how ransiions occur among he belief saes. We define b a o = b a o s, where b a o s represens he probabiliy of occupying sae s S PO a ime + 1, given ha he decision maker s belief abou he paien s healh sae was b, acion aken was a, and observaion seen was o a ime. Tha is, b a o s = Pr s b a o. Alhough b a o depends on ime, we drop his ime index for he clariy of he noaion. Our modeling framework is differen han he convenional POMDP models in wo ways. Firs, he order of

6 Ayer, Alagoz, and Sou: Personalized Mammography Screening 1024 Operaions Research 605), pp , 2012 INFORMS evens is differen. In our model, a he beginning of each decision epoch, an acion is aken, an observaion is seen and, depending on he oucome of he observaion, he ransiion o a new sae occurs Figure 2). Whereas in a convenional POMDP model formulaion Smallwood and Sondik 1973) an acion is aken, he ransiion occurs based on his acion, and hen an observaion is seen, i.e., he observaion is seen afer he ransiion occurs. Second, in our model, unlike he convenional POMDP models, sae ransiions depend no only on he seleced acion bu also on he observaion seen. For example, upon experiencing a posiive mammogram, if he woman has cancer, she sars reamen before a ransiion occurs and he decision process ends here sopping condiion). This modeling framework fis beer, especially in diagnosic decisions, in which depending on he oucome of he observaion, he follow-up procedure may change he sae ransiions e.g., a biopsy migh resul in reamen). Noe ha when here is no sopping condiion, i is possible o ransform our POMDP ino an equivalen classical POMDP see he elecronic companion for a proof). These differences change he beliefsae updae equaions as well as he opimaliy equaions. We compue he updaed belief sae as follows: b a o s s S PO b s Ka a o o s P s s s S PO b s Ka o s = if a=w o W or a=m o =M 1) P M M+ s 0 if a=m o =M + When a = W o W or a = M o = M, he compuaion of b a o is sraighforward. When a = M and o = M+, if he paien is found o have cancer s = 1 or s = 2) afer a posiive mammogram M+), here is no updae o he belief sae because he reamen sars and he decision process ends. On he oher hand, when a = M and o = M+, if he paien is found o be cancer free s = 0), i.e., experiences a false-posiive mammogram, hen his paien coninues o follow he decision process. R s : Toal expeced poscancer QALYs accrued a ime when he paien is in one of he cancer saes s 1 2 ), deeced by biopsy, and has sared cancer reamen. r s a o : Expeced QALYs beween ime and + 1 when he paien s rue healh sae is s, he acion chosen is a, and he observaion seen is o. The funcion r s a o incorporaes life expecancy beween ime and + 1 and disuiliies associaed wih a mammogram ha occurs in ha ime inerval, esimaion of which is explained in 3.2. Noe ha if he paien is in one of he cancer saes, s 1 2 ) and experiences a posiive mammogram, hen she is assigned a lump-sum reward QALYs), R s. Tha is, no QALYs are assigned over he nex decision epoch upon experiencing a rue posiive mammogram, i.e., r 1 M M+ = r 2 M M+ = 0. r s a : Expeced QALYs beween ime and + 1 when he paien s rue healh sae is s and he acion chosen is a, which is compued by r s a = o a K a o s r s a o. r T s : Toal expeced remaining QALYs a ime T given ha he paien is alive a ime T and her rue healh sae is s Opimaliy Equaions Le V and V b represen he maximum oal expeced QALYs he paien can aain when he curren informaion sae is S and he curren belief sae is b B S PO a ime, respecively. Then, R 1 if = =max R 2 if = V b if b = oherwise. 2) V Also, le V a b represen he maximum oal expeced QALYs he paien can aain upon aking acion a when he curren belief sae is b B S PO a ime. Then, V W b = max V b V M b where V W b = b s K W o s r s W o s S PO o W + P s S V M b = [ b s K M s S PO W o s s V +1 ) b W o and M s r s M M + )] M M P s s V +1 b M M s S + b 0 K M M+ 0 r 0 M M+ + ) M M+ P s 0 V +1 b M M+ s S 2 + b s K M M+ s R s = 0 T 1 s=1 Moving he funcions ha do no depend on s ou of he summaion over s S and noing ha a o s S P s s = 1 for all s S PO, a A, and o, we obain V b { =max b s K W o s r s W o +V +1 b W o ) s S PO o W b s [ K M M s r s M M s S PO +V+1 b M M )]

7 Ayer, Alagoz, and Sou: Personalized Mammography Screening Operaions Research 605), pp , 2012 INFORMS b 0 K M M+ 0 r 0 M M+ +V +1 b M M+ ) } 2 + b s K M M+ s R s =0 T 1 and, 3) s=1 V T b = s S PO b s r T s 4) 2.2. An Alernaive Form for he Opimaliy Equaions In his secion, we presen an equivalen represenaion for he opimaliy equaions ha is used for developing soluion algorihms and is easier o inerpre. We sar wih he following lemma, which shows ha he opimal value funcion is piecewise linear and convex, upon which he alernaive represenaion is buil. The proofs of all resuls in his secion are presened in he elecronic companion. Lemma 1 Smallwood and Sondik 1973). The opimal value funcion V b is piecewise linear and convex for all T, and hence can be expressed as he maximum of a finie number of linear funcions. Tha is, { } V b = max b s k s k s S PO where i = i s s S PO for some = 0 1 5) are called he -vecors. We nex presen he opimaliy equaions in erms of he -vecors. Proposiion 1. The opimal value funcion V b can be equivalenly represened as follows. V b b s s S PO b s K M s S PO = max + + o W K W P W o s S PO [ o s r s W o s b W o s +1 s [ M s r s M M P M M s S PO ] s b M M s W) ] +1 s [ + b 0 K M M+ 0 r 0 M M+ + max k P M M+ s 0 k +1 s s S PO + b 1 K M M+ 1 R 1 + b 2 K M M+ 2 R 2 M) )] 6) where b a o = arg max k { b s K a o s s S PO P a o s S PO } s s k +1 s 7) Noe ha b a o depends on ime ; however, we drop his ime index for he clariy of he noaion. The following lemma presens he explici represenaion of he -vecors, upon which he soluion algorihms are buil. l b a Lemma 2. Le denoe he maximizing -vecor for belief sae b and given acion a, and le l b denoe he opimizing -vecor for belief sae b. Then, l b W l b M = and K M s = o W K W o s r s W o + P W o s S PO s [ M s r s M M + P M M s S [ PO + K M M+ s r s M M+ + max k K M l b = arg max k [ M s r s M M + s b W o s +1 s s b M M s ) 8) ] +1 s P M M+ s s k +1 s s S PO if s = 0 P M M s S PO s b M M s + K M M+ s R s if s = 1 or s = 2 = arg max l b W l b M { } b s k s s S PO { b s l b W s s S PO b s l b M s S PO )] ] +1 s } s 9) 10) For our problem, l b s can be inuiively inerpreed as he maximum QALYs ha a paien wih belief sae b can obain when her rue healh sae is s a ime. We furher srucurally analyze his POMDP and presen reasonable sufficiency condiions ha ensure he exisence

8 Ayer, Alagoz, and Sou: Personalized Mammography Screening 1026 Operaions Research 605), pp , 2012 INFORMS of srucured value funcions and policies in he elecronic companion. In paricular, we firs show ha he sufficiency condiions used in previous POMDP models for he monooniciy of he opimal value funcion in belief vecor b are no saisfied in our case, and provide new sufficiency condiions. Second, we show under cerain assumpions ha he opimal policy is of conrol-limi ype. Tha is, if he opimal acion for a relaively healhier sicker) paien is Mammography Wai), hen he opimal acion for a sicker healhier) paien should also be Mammography Wai). Unforunaely, he assumpions menioned depend on he compued parameers from a previous ieraion, and hence do no provide a way o predic a conrol-limi policy before ha policy is acually compued. Neverheless, we believe hese resuls shed useful insighs ino he overall problem. 3. Model Inpus In his secion, we firs presen our sources of model inpus in 3.1, hen describe selecion of model inpus in 3.2, and finally in 3.3 we presen models for esimaing breas cancer risk, which is necessary o implemen he POMDP model Sources of Model Inpus Our primary source of model inpus is a validaed microsimulaion model of breas cancer epidemiology in he Unied Saes developed a he Universiy of Wisconsin, Madison, which we refer o as he Universiy of Wisconsin Breas Cancer Simulaion UWBCS). The UWBCS was developed as par of he Cancer Inervenion and Surveillance Modeling Nework CISNET), a Naional Cancer Insiue NCI)-sponsored consorium focusing on saisical modeling of he impac of cancer conrol inervenions and opimal cancer conrol planning see Some of he CISNET breas cancer models, including he UWBCS, have been used o invesigae populaion-based mammography-screening sraegies Sou e al. 2006, Mandelbla e al. 2009). In fac, CISNET models, including he UWBCS, provided evidence for he recen USPSTF recommendaions USPSTF 2009), which have reignied he conroversy abou mammography screening and had an exensive media coverage New York Times 2009, USA Today 2009, CNN 2009, Washingon Pos 2009). The UWBCS is a highly deailed simulaion model designed o replicae breas cancer naural hisory, deecion, reamen, and moraliy raes in he U.S. populaion. The saes of he UWBCS model include cancer-free, in siu, localized invasive, regional invasive, and disan invasive cancers. The model is able o replicae populaionlevel U.S. cancer surveillance daa by simulaing he individual life hisories of women aged 20 years or older in proporion o heir prevalence in he U.S. populaion. The UWBCS is informed by NCI-provided inpus common o all CISNET models and calibraed o breas cancer incidence daa repored by he NCI s Surveillance, Epidemiology, and End Resuls SEER) program and breas cancer moraliy daa repored by Naional Cener for Healh Saisics using he accepance sampling mehod. The model is furher cross validaed agains he Wisconsin Cancer Reporing Sysem WCRS). Deailed descripions of model design, assumpions, and validaion have been published elsewhere Fryback e al. 2006) Selecion of Inpu Parameers Sources of model inpus are lised in Table 2. We specify he age-specific core sae ransiion probabiliies represening he naural hisory of breas cancer and expeced QALYs a each decision epoch i.e., inermediae rewards) using he UWBCS model. We combine he localized, regional, and disan invasive cancer saes in he UWBCS ino a single invasive cancer sae. To specify he ransiion probabiliies ha represen he naural hisory of breas cancer, we exclude screening from he simulaion model and calculae he model predicions ha represen individual life hisories of over 22 million women. We specify he age-specific ransiion probabiliy p ij as he proporion of ransiions from sae i ha end up in sae j. We validaed ha he POMDP model informed by hese ransiion probabiliies is able o reproduce he U.S. populaion under no screening scenario. To specify he oal expeced QALYs before a cancer deecion during each decision epoch, i.e., he inermediae rewards r s W and r s M M, we employ he half-cycle correcion mehod Sonnenberg and Beck 1993) using age-and sae-specific moraliy raes from he ransiion probabiliies and disuiliy values of mammography. Specifically, o specify he oal QALYs a he curren decision epoch when he acion aken is Wai, we assign 0.5 life years if he paien is alive in he curren decision epoch, and 0.25 life years if he paien dies in he curren decision epoch. Tha is, r s W = 0 5 Palive in he curren decision epoch curren healh sae is s Pdies in he curren decision epoch curren healh sae is s. To specify he inermediae rewards when he acion aken is Mammmography, we subrac he disuiliy values associaed wih a mammogram from he inermediae rewards gained when he acion aken is Wai. Tha is, if we le du M o s denoe he disuiliy associaed wih Table 2. Parameer Sources of model inpus. Daa source Core sae ransiion probabiliies UWBCS Inermediae rewards UWBCS Lump-sum rewards SEER Sensiiviy and specificiy Kerlikowske e al. 2000) of mammography Sensiiviy and specificiy of CBE Baron e al. 1999) Sensiiviy and specificiy of BSE Baxer 2001) CBE proporion in he populaion Elmore e al. 2005) BSE proporion in he populaion Messina e al. 2004)

9 Ayer, Alagoz, and Sou: Personalized Mammography Screening Operaions Research 605), pp , 2012 INFORMS 1027 a mammogram wih oucome o for he rue healh sae s, hen r s M M = r s W du M M s for all s S PO and r 0 M M+ = r 0 W du M M+ 0. In our base-case analysis, we use he following values for he disuiliy associaed wih mammography: a) 0.5 days for a negaive mammogram Mandelbla e al. 1992), b) wo weeks for a rue posiive mammogram Velanovich 1995), and c) four weeks for a false-posiive mammogram, because he lieraure repors ha he disuiliy for a falseposiive mammogram is higher han ha for a rue-posiive mammogram Earle e al. 2000). We assume ha mammograms do no increase he fuure risk of cancer, because he ACS repors ha his risk is minimal ACS 2009). We specify he poscancer life expecancies i.e., he lumpsum rewards R s using age-specific moraliy raes for paiens under cancer reamen from he SEER daa Jemal e al. 2009), based on he mehod described in Arias 2006). We draw from he lieraure o specify sensiiviy and specificiy of exams mammography, BSE, CBE), which generae he observaion probabiliy marices. Specifically, we obain age-specific sensiiviy and specificiy of mammography separaely for in siu and invasive cancers from Kerlikowske e al. 2000), sensiiviy and specificiy of CBE from Baron e al. 1999), and sensiiviy and specificiy of BSE from Baxer 2001) Table 3). To specify he sensiiviy and specificiy of self-deecion, we compue he weighed average of CBE and BSE sensiiviy and specificiy values by using he CBE and BSE proporions in he populaion repored by Elmore e al. 2005) and Messina e al. 2004) Table 2). In he following secion we describe how we esimae he risks of in siu and invasive cancer a age 40, which are necessary o implemen he opimal screening sraegy obained by he POMDP model Risk Esimaion Models The esimaed risks of in siu and invasive cancer are used o calculae he iniial belief saes a age 40. The Table 3. Sensiiviy and specificiy values of ess used in he POMDP model. a) Mammography Age group Sensiiviy %) Specificiy %) b) BSE, CBE, and self-deecion Tes Sensiiviy %) Specificiy %) BSE CBE Self-deecion risks a laer ages can be calculaed by using hese iniial belief saes and screening hisory via he belief updae Equaion 1). There are several risk esimaion models for in siu and invasive breas cancers ha could be used for his purpose see, for example, Reinier e al. 2007, Claus e al. 2001, Trenham-Diez e al. 2000). To find he risks a age 40, we use a modified version of he Gail model, which is a validaed model for predicing breas cancer risk in individual paiens in daily clinical pracice, and he mos commonly used risk esimaion model in he medical communiy Cosanino e al. 1999). The Gail model esimaes he invasive breas cancer risk for an individual based on he following risk facors: curren age, age a menarche, age a firs live birh, number of firs-degree relaives wih breas cancer, number of previous biopsies, and presence of aypical findings on a biopsy. The iniial Gail model was primarily based on a Caucasian populaion and provided he esimaion for oal risks of in siu and invasive risks. Laer modificaions o his model allowed applicabiliy o African American women and esimaing he risk specifically for invasive breas cancer Cosanino e al. 1999, Gail e al. 1999). To esimae he in siu cancer risk, Gail e al. 1999) proposed using he incidence raio beween in siu and invasive cancers. The modified version of he Gail model is available on he NCI s websie hp:// and is used 20,000 o 30,000 imes each monh o quanify an individual woman s risk of breas cancer Elmore and Flecher 2006). 4. Compuaional Experimens We solve our POMDP model opimally using Monahan s algorihm Monahan 1982) wih Eagle s reducion Eagle 1984) see he elecronic companion for he algorihm). We used an Inel Xeon 2.33 GHz processor wih 16 GB RAM for our compuaional experimens. The compuaion ime for solving he base case using our soluion algorihm was hours Opimal Screening Sraegy Figure 3 depics he opimal screening sraegy as a funcion of in siu and invasive breas cancer risks for various ages. In Figure 3, he horizonal and verical axes represen he probabiliy risk) of in siu and invasive cancer, respecively; herefore, he probabiliy of being cancer free is equal o 1 minus hese probabiliies e.g., if he risk of in siu cancer = and he risk of invasive cancer = 0 003, hen he probabiliy of being cancer free = = 0 996). As shown in Figure 3, here exiss hreshold risks of in siu and invasive cancers beyond which he opimal acion is Mammography, as proven in Theorem 2 elecronic companion). In addiion, he mammographyscreening hreshold risk for invasive cancer is lower han ha of he in siu cancer. In oher words, he risk of invasive

10 Ayer, Alagoz, and Sou: Personalized Mammography Screening 1028 Operaions Research 605), pp , 2012 INFORMS Figure 3. Opimal mammography-wai decisions for paiens a differen ages. Risk of invasive cancer year-old paiens Risk of in siu cancer Risk of invasive cancer year-old paiens Risk of in siu cancer Risk of invasive cancer year-old paiens Risk of in siu cancer Risk of invasive cancer year-old paiens Risk of invasive cancer year-old paiens Mammogram Wai Risk of in siu cancer Risk of in siu cancer cancer is more significan han he risk of in siu cancer in deermining he mammography decision, as proven in Corollary 2 elecronic companion). Furhermore, he area for which he Wai acion is opimal increases as he paien ages, i.e., he hreshold risk for he Mammography acion is higher in older women. Noe ha because he risk of breas cancer also increases wih age, we canno direcly inerpre his finding because older paiens are less likely o be recommended for mammography. Insead, he mammography decision is deermined by he rade-off beween hese wo facors, as illusraed in he following case examples. Case 1. We consider a 40-year-old whie woman who has no personal or family hisory of breas cancer or a hisory of a previous biopsy, began mensruaion a age 14, and her firs birh was a age 23. Using he Gail model described in 3.3, we esimae his paien s curren risks of in siu and invasive cancers as 0.1% and 0.2%, respecively. Based on our model, we find ha he opimal acion for his paien is o Wai, as opposed o he recommendaions by some populaion-based guidelines Figure 3). Case 2. We now consider a 50-year-old whie woman who has he same risk facors as he woman in Case 1. We know ha his paien did no have any prior mammograms. Using he Gail model and belief updae equaion in 1), we esimae his paien s curren risks of in siu and invasive cancers as 0.6% and 0.7%, respecively. Based on our model, we find ha he opimal acion for his paien is o have a Mammmogram, as recommended by populaionbased guidelines Figure 3). Our POMDP model could also be used o find he opimal screening inervals, depending on he oucome of he observaions. For example, for he paien in Case 1, assuming ha no self-deecion occurs in he following years, her nex mammography screening should be scheduled for age 42. Similarly, for he paien in Case 2, if he oucome of he mammogram a age 50 is negaive and no self-deecion occurs in he following years, her nex mammography screening should be scheduled for age 53. Similarly, we can find he opimal screening inervals for all possible belief, acion, and observaion scenarios. Below, we provide wo examples o illusrae how risk sraificaion affecs screening recommendaions and how screening inervals may change afer years of negaive mammograms. Case 3. This example illusraes how risk sraificaion affecs screening recommendaions. Suppose we have a paien a age 40, whose in siu and invasive cancer risks are esimaed as and , respecively. Then, based on he opimal screening policy presened in Figure 3, he opimal acion for his paien is o undergo mammography a age 40. Assuming ha he oucomes of his and he following mammograms are all negaive and no self-deecion occurs, his paien should undergo mammography exams a ages 40, 44, 48, 51, 55, 58, 62, 66, and 72. The updaed risks and he corresponding opimal acions for his paien are presened in Table 4. Case 4. This example illusraes how screening inervals may change afer years of negaive mammograms. Suppose he same paien in Case 3 undergoes annual mammography exams beween ages including age 55), all of which urn ou o be negaive. Using he belief updae equaion Equaion 1)), we esimae her updaed risk a age 56 as 0.25% for in siu cancer and 0.13% for invasive cancer. Then, our resuls show ha he nex mammogram should be scheduled for age 60. Tha is, he screening inerval increases from 3 years o 5 years afer 15 years of annual

11 Ayer, Alagoz, and Sou: Personalized Mammography Screening Operaions Research 605), pp , 2012 INFORMS 1029 Table 4. The effecs of risk-sraificaion on screening recommendaions Case 3). Risk of Risk of Opimal Age in siu cancer invasive cancer acion M W W W M W W W M W W M W W W M W W M W W W M W W W M W W W W W M W W W W W W W W mammograms see in Case 3 ha if a woman is screened a age 55, he nex screening should be scheduled for age 58). For his specific paien, he updaed risks and he opimal acions are given in Table 5. A woman s prior screening hisory may significanly influence he esimaed risk of breas cancer Elmore e al. 2005), and in urn migh change he opimal acion. Table 5. The effecs of negaive mammograms on screening inervals Case 4). Risk of Risk of Opimal Age in siu cancer invasive cancer acion M W W W W M Figure 4 shows how esimaed breas cancer risk changes under differen screening scenarios. Specifically, we compare hree scenarios: no screening, screening biennially afer age 40, and screening biennially afer age 50, assuming ha each screening resul was negaive. To illusrae, consider wo differen paiens who are boh 55 years old wih he same risk characerisics, bu differen screening hisories: paien 1 has never had a screening mammogram; paien 2 had prior mammograms a ages 50, 52, and 54, all of which were negaive. Then, as shown in Figure 4, whereas paien 1 has a 2.6% oal risk of breas cancer 1.4% invasive, 1.2% in siu), paien 2 has only 0.53% oal risk 0.19% invasive, 0.34% in siu). Whereas he opimal acion for paien 1 is o have a Mammogram, he opimal acion for paien 2 is o Wai a he curren ime period. This suggess ha incorporaing personal hisory of screening ino screening recommendaions may reduce unnecessary mammograms and help make beer screening decisions Acual Screening Policies vs. he Opimal Screening Sraegy In his secion, we compare he performances of no screening and several acual populaion-based screening guidelines presened in Table 1 o our proposed opimal screening sraegy. When he age o end screening is no specified by an acual screening policy, we se he ending age o eiher 74 or 79, because hese are he commonly acceped ending ages Mandelbla e al. 2009). Specifically, we compare he oal expeced QALYs, he number of mammograms, he number of false posiives, and he lifeime risk of developing undeeced invasive cancer by each screening scenario for an average-risk and a high-risk paien a age 40. We define a high-risk paien as someone who has a family hisory of breas cancer. We esimae he cancer risks for average-and high-risk women using he risk esimaion model described in 3.3. We assume ha if he paien has a deeced cancer a any ime period, she is reaed and kep under surveillance wih annual mammograms unil age 80, regardless of he policy adoped. Table 6 presens our resuls. Furhermore, Figures 5 and 6 illusrae he socieal and personal rade-offs based on resuls presened in Table 6 and idenify efficien froniers. In Table 6, when calculaing he expeced number of mammograms and false posiives, we assume ha women wih a personal hisory of breas cancer will have he same moraliy raes wih hose women wihou any personal hisory, because very lile is known abou he moraliy raes of such women. For an average-risk paien, he opimal screening sraegy reduces he expeced number of mammograms by more han 14, reduces he number of false-posiive mammograms a leas by half, and increases he oal expeced QALYs by a leas 2.3 monhs, compared o annual screening beween ages Alhough he reducions in he expeced number of mammograms and false posiives are remarkable, he savings in oal

12 Ayer, Alagoz, and Sou: Personalized Mammography Screening 1030 Operaions Research 605), pp , 2012 INFORMS Figure 4. Breas cancer risk as a funcion of personal hisory of screening. Toal risk of breas cancer No screening Screening every wo years No screening beween 40 49, screening every wo years afer 50 Paien 1 Paien Age expeced QALYs may appear o be nonsignifican. However, as Wrigh and Weinsein 1998) highligh, he gains of monhs in QALYs from screening are equivalen o gains of years from breas cancer reamen. Furhermore, hese savings increase wih increased risk. There appears o be lile benefi in screening women afer age 74 in erms of he QALY gains. On he oher hand, according o our model, screening women beween 40 49, an age group where benefis of mammography are paricularly conroversial, does provide significan QALY gains, especially for high-risk women due o a longer life expecancy Table 6). In erms of lifeime risk of developing an undeeced invasive cancer an invasive cancer ha is no deeced by any mammograms during he woman s lifeime), aggres- sive screening policies wih earlier saring ages such as annual screening beween 40 and 79) perform beer han he oher recommended policies, as expeced. For many cases including boh average and high-risk women), he opimal screening sraegy resuls in a lower lifeime risk of developing undeeced invasive cancer han he various recommended screening policies. Alhough he opimal screening sraegy leads o a higher lifeime risk of developing undeeced invasive cancer han some of he aggressive screening policies such as annual screening beween 40 and 79), his difference is significanly smaller when compared o he difference beween no screening and he opimal policy. Furhermore, he difference in lifeime risk of developing undeeced invasive cancer does no necessarily imply he same magniude of difference in life years, Table 6. Comparison of various screening sraegies for a 40-year-old average or high-risk paien. Expeced number Expeced number Lifeime risk of developing Screening of mammograms of false-posiives undeeced inv. cancer %) Sar End inervals age age in years) Average-risk High-risk Average-risk High-risk Average-risk High-risk Expeced QALYs No screening %) %) %) %) %) %) %) %) %) %) %) %) %) %) %) %) %) %) %) %) %) %) Opimal sraegy %) %) Expeced lifeime risk of developing undeeced invasive cancer. Numbers in parenheses show he percenage increase when compared o no screening.

13 Ayer, Alagoz, and Sou: Personalized Mammography Screening Operaions Research 605), pp , 2012 INFORMS 1031 Figure 5. Socieal rade-off represenaion of he screening policies repored in Table 6. Expeced QALYs Average-risk women 37.5 Expeced QALYs High-risk women Expeced number of mammograms Expeced number of mammograms None ) ) ) ) ) ) ) ) ) ) ) Opimal Efficien because 1) undeeced invasive cancers do no always lead o moraliy, and 2) deahs due o undeeced invasive cancers a older ages migh no make a significan difference in life years. To assess how hese differences in lifeime risk of developing undeeced invasive cancers ranslae ino life years, we calculaed life years in addiion o QALYs) associaed wih annual screening beween ages which resuls in he minimum lifeime undeeced invasive cancer risk) and he opimal screening sraegy. The difference in life years beween he annual screening policy beween ages and he opimal screening sraegy was very small for average-risk women around years, or equivalenly 2 days). Furhermore, for high-risk women, he opimal screening sraegy performed slighly beer han he annual screening beween ages in erms of life years versus ). This is because he opimal screening sraegy has he power o require biannual screening a imes if needed, which migh be useful, especially a younger ages. In fac, separae analyses showed ha, for high-risk women, he opimal screening sraegy asks for very aggressive biannual) screening a younger ages unil eiher 1) an exising cancer is deeced, or 2) several sequenial negaive mammograms are observed, which significanly decreases he woman s risk of breas cancer. For example, for a 40-year-old high-risk woman, he opimal sraegy prescribes four sequenial mammograms unil age 42 a ages 40, 40.5, 41, and 41.5). Anoher measure for screening efficiency would be he delay in cancer deecion a early sages; however, our model does no include cancer sages and hence does no allow compuaion of such a saisic. To assess he robusness of he resuls presened in his secion, we conduc sensiiviy analyses on inpu parameers, which show ha he resuls are sensiive o he disuiliies associaed wih a mammogram and subsanially robus agains small changes in he accuracy of mammography see he elecronic companion) Value of Self-Deecion To invesigae he effec of self-deecion in early diagnosis of breas cancer, we compare he oal expeced QALYs and number of mammograms obained by he opimal screening Figure 6. Personal rade-off represenaion of he screening policies repored in Table 6. Lifeime risk of undeeced invasive cancer %) Average-risk women 2 Expeced number of false-posiives 3 4 Lifeime risk of undeeced invasive cancer %) High-risk women 2 Expeced number of false-posiives 3 4 None ) ) ) ) ) ) ) ) ) ) Opimal Efficien