Quantification of population benefit in evaluation of biomarkers: practical implications for disease detection and prevention

Transcription

1 Li et al. BMC Medial Informatis and Deision Making 2014, 14:15 CORRESPONDENCE Open Aess Quantifiation of population benefit in evaluation of biomarkers: pratial impliations for disease detetion and prevention Xiaohong Li 1,2*, Patriia L Blount 1,3, Brian J Reid 1,2,3,5 and Thomas L Vaughan 1,4 Abstrat Bakground: With the rapid development of -omi tehnologies, an inreasing number of purported biomarkers have been identified for aner and other diseases. The proess of identifying those that are most promising and validating them for use at the population level for prevention and early detetion is a ritial next step in ahieving signifiant health benefits. Methods: In this paper, we propose that in order to effetively translate biomarkers for pratial linial use, it is important to distinguish and quantify the differenes between the use of biomarkers and other risk fators to identify preventive interventions versus their use in disease risk predition and early detetion. We developed mathematial models for quantitatively evaluating risk and benefit in use of biomarkers for disease prevention or early detetion. Simple numerial examples were used to demonstrate the potential appliations of the models for various types of data. Results: We propose an index whih takes into aount potential adverse onsequenes of biomarker-driven interventions the naïve ratioofpopulationbenefit(rpb) to failitate evaluating the potential impat of biomarkers on aner prevention and personalized mediine. The index RPB is developed for both binary and ontinuous biomarkers/risk fators. Examples with omputational analyses are presented in the paper to ontrast the differenes in using biomarkers/risk fators for prevention and early detetion. Conlusions: Integrating epidemiologi knowledge into linial deision making is a key step to translate new biomarkers/risk fators into pratial use to ahieve health benefits. The RPB proposed in this paper onsiders the absolute risk of a disease in intervention, and takes into aount the risk-benefit effets simultaneously for a marker/exposure at the population level. The RPB illustrates a unique approah to quantitatively assess the risk and potential benefits of using a biomarker/risk fator for intervention in both early detetion and prevention. Keywords: Ratio of population benefit, RPB, Biomarkers, Disease prevention, Disease early detetion, Clinial deision making, Biomarkers for early detetion, Risk/benefit analysis Bakground The identifiation of robust aner risk fators and biomarkers are the ornerstones of modern approahes to aner prevention and personalized mediine. A large number of environmental and host risk fators (either inherited or somati) have been identified that are assoiated with aner risk, and with rapidly-advaning * Correspondene: xili@fhr.org 1 Divisions of Publi Health Sienes, Fred Huthinson Caner Researh Center, 1100 Fairview Ave. N., Seattle, WA 98109, USA 2 Human Biology, Fred Huthinson Caner Researh Center, 1100 Fairview Ave. N., Seattle, WA 98109, USA Full list of author information is available at the end of the artile -omis tehnologies, the reported number of biomarkers proposed for linial use is inreasing dramatially. However, the translation of these for use in the population or lini in suh a way as to have a signifiant impat on aner inidene and mortality is still a major hallenge. The proess of seleting and evaluating the most promising biomarkers for linial appliation among the large number of purported biomarkers is a ritial step in the translation proess. Key to this proess is distinguishing the differenes between evaluating biomarkers and risk fators for primary prevention programs versus disease risk predition and 2014 Li et al.; liensee BioMed Central Ltd. This is an Open Aess artile distributed under the terms of the Creative Commons Attribution Liense ( whih permits unrestrited use, distribution, and reprodution in any medium, provided the original work is properly redited. The Creative Commons Publi Domain Dediation waiver ( applies to the data made available in this artile, unless otherwise stated.

2 Li et al. BMC Medial Informatis and Deision Making 2014, 14:15 Page 2 of 12 early detetion. Quantitative analysis of these differenes an failitate the translational proess. Pepe et al. [1] ompared the assoiation of a marker with a disease, often quantified in ase-ontrol or rosssetional studies by the odds ratio (OR), with use of the marker for disease lassifiation (i.e. presene or absene of aner in a sample), and illustrated the limitation of the OR in gauging the performane of a diagnosti, prognosti, or sreening marker. More reently, the use of markers disovered in geneti assoiation studies for disease risk predition was speifially addressed by Jakobsdottir et al. [2]. The limitations of using markers for medial diagnosis or early detetion also have been omprehensively assessed [3-5]. Reently, an inreasing number of studies have foused on the use of previouslyidentified risk fators and biomarkers (e.g., one or more onstitutive SNPs (single-nuleotide polymorphism)) for aner prevention or for pathway-targeted therapy development [6,7]. In pratie, the speifi riteria for evaluating a biomarkerorriskfatorfordiseasedetetion/predition ould be quite different than that for disease prevention. To highlight and evaluate these differenes quantitatively, we first illustrate the numerial relationship between the OR of a biomarker/risk fator and its population attributable risk perent (PAR%) (assuming ausality) in the ontext of a population prevention program. We then illustrate the orresponding auray, as measured by sensitivity and speifiity, for a biomarker/risk fator with idential harateristis (OR and prevalene) in the ontext of a disease detetion/predition program. Finally we propose an index the naïve ratio of population benefit (RPB) for quantifying overall risk/benefit of using a biomarker for aner prevention or detetion/predition at the population level. Analyses are presented separately for binary and ontinuous biomarkers. Methods and results Numerial relationships between sensitivity, speifiity and population attributable risk for binary and ontinuous biomarkers Calulation for binary marker/risk fator The PAR% is often used to estimate the fration of the total disease burden in the population that would not have ourred if a ausal risk fator were absent [8]. To help introdue the latter parts of the paper, we first illustrate the numerial relationships between PAR% for ausal binary markers/risk fators at different prevalene and relative risk levels and the orresponding sensitivity and speifiity of using a marker/risk fator with idential harateristis for disease lassifiation/predition (Table 1). The status of a speifi binary risk fator (e.g. a mutated gene or exposure) and the observed disease outome status, also binary, an be displayed as a standard 2 2 ontingeny table, with the four ells labeled a(+/+), b(+/-), (-/+) and d(-/-) orresponding to the ounts of individuals in a ohort with status of exposure and outome (+ for yes, - for no), respetively. If outome is diretly predited by the marker, for alulation of sensitivity and speifiity (either for sreening or sreening-based disease intervention), the data an be arranged in an idential 2 2 table with the individual ells labeled a (true positive), b (false positive), (false negative) and d (true negative) relating the biomarker status with a true outome status or gold standard. Using the ounts in the four ells of the ontingeny table (whether orresponding to exposure and outome or disease lassifiation) several ommonly used quantities an be obtained. A binary marker/risk fator has two possible values, leading to fixed sensitivity (a/(a+)) and speifiity (d/(b+d)) values in a population with a speifi OR and marker prevalene, in whih the inidenes in exposed (or marker arriers) and unexposed (or marker non-arriers) are a/(a + b), /( + d) respetively. Table 1 shows the numerial relationships among i) prevalene of a risk fator/marker, ii) the relative risk of disease assoiated with the marker (indiated by OR), iii) PAR%, iv) sensitivity, and v) speifiity. If we let r 1 =((a+)/(a+b+ +d)), r 2 = /(+d), (r 1 is the prevalene and r 2 is the false negative fration), the population attributable risk an be alulated as PAR% = (r 1 -r 2 )/r 1 100%. By definition, the false positive fration = 1-speifiity; the false negative fration = 1-sensitivity; and the OR = (sensitivity/(1- sensitivity))/((1-speifiity)/speifiity). In Table 1, marker prevalene and risk fator exposure prevalene are interhangeable algebraially; the former used for early detetion and risk predition, and the later used for prevention. The alulation of PAR% above was based on the assumption of no adjustment for potential onfounders. A ommon way to obtain PAR% adjusted for onfounders is to use a stratifiation approah: PAR adj % ¼ X p i PAR i % i where p i is the proportion of ases in stratum i, PAR i % is the PAR% estimated from stratum i. More details for dealing with onfounders an be found in Rothman et al. [9]. Numerial analysis for ontinuous marker/risk fator Pepe et al. [1] evaluated the limitation of the OR in gauging the performane of a diagnosti, prognosti, or sreening marker. Illustrated here is the use of ontinuous biomarkers both for diagnosti/prognosti/sreening and for prevention, along with the relationship between the OR value and PAR% parameters for the ontinuous markers. Figure 1 presents a few hypothetial normal distributions of ontinuous markers/risk fators with different OR risk values. For the ontinuous distribution markers, the sensitivity and speifiity an be alulated as follows [10],

3 Table 1 Numerial illustration for alulating RPB of hypothetial binary markers using data of three aners as examples Biomarker/exposure harateristis Ratio of population benefit (RPB) Net benefit (NB) OR Prevalene (%) PAR Sen. Spe. RPB RPB RPB NB NB NB % (EA, f 0.03) (Breast a, f 0.06) (Ovarian a, f 0.15) (EA, f 0.03) (Breast a, f 0.06) (Ovarian a, f 0.15) Li et al. BMC Medial Informatis and Deision Making 2014, 14:15 Page 3 of 12

4 Table 1 Numerial illustration for alulating RPB of hypothetial binary markers using data of three aners as examples (Continued) a= aner. EA= esophageal adenoarinoma. f= loss adjustment fator of quality-adjusted life year. NB= net benefit. OR= odds ratio. The table shows numerial relationship between Odds ratio, Marker prevalene, PAR% of binary markers and their RPB based on the aner data of three studies. NB were also alulated. The table assumes 1% disease prevalene in general population. For PAR%, sensitivity and speifiity similar patterns are observed with other disease prevalene values less than about 10%. Disease prevalene affets RPB more diretly (see text). Li et al. BMC Medial Informatis and Deision Making 2014, 14:15 Page 4 of 12

5 Li et al. BMC Medial Informatis and Deision Making 2014, 14:15 Page 5 of 12 Figure 1 Hypothetial distribution patterns of ontinuous markers with different relative risks, and thresholds for risk predition. Two normal distributions (mean = 0 and standard deviation = 0.5) are used to represent the distribution of a ontinuous marker in disease (solid urved line) and non-disease (dashed urved line) populations for six different ORs. The loations of the means for the disease population are onsistent with the logit model Pr(D=1 X)=α + βx, in whih one unit inrease orresponds to the OR shown in the figure. The three vertial bars (solid, dotted, and dashed) orrespond to different thresholds (ut off value ) for positive-negative alls of a disease with a ontinuous distribution marker. Speifially, the solid bar represents the threshold value suh that the sensitivity is kept for 0.95 for various OR values in the plot; the dotted bar represents the threshold value suh that the sensitivity and speifiity are equal for various OR values in the plots; and the dashed bar represents the threshold value suh that the speifiity is kept for 0.95 for various OR values in the plot. The examples of using the ontinuous marker for disease lassifiation or prevention are shown in Figure 2; and orresponding sensitivity, speifiity and PAR% of various thresholds (three bars in this Figure) are shown in Table 2 and Figure 2 (the ross, irle, and triangle in Figure 2 orrespond to solid, dashed, and solid vertial bars in Figure 1). Sensitivity ¼ PY ð D > Þ ¼ Φ μ D σ D ; Speifiity ¼ 1 P Y D > ¼ 1 Φ μ D σ,where D indiates non-disease group, and D isthethresholdabovewhihapositive(disease)all will be made. In ontrast to binary markers, whih only have one set of sensitivity and speifiity values, ontinuous markers an be used to generate infinite sets of sensitivity and speifiity values depending on the threshold value of. To quantify PAR% for ontinuous markers, let w be the proportion of diseased individuals in a population or risk of a disease in the general population, then for a marker with a ontinuous value, a speifi set of sensitivity and speifiity is obtained for a given threshold, theriskof unex- posed (the proportion of subjets, either diseased or nondiseased whose marker level is lower than the threshold ) q ue with threshold an be alulated as q ue ¼ R w½1 f d R ðþdx x R ð1 wþ f d ðþdxþw½1 x f d ðþdx x ¼ wð1 sensitivity Þ ð1 wþspeifiity þwð1 sensitivity Þ, where f d (x) andf dx ðþare the probability density distribution of a biomarker in the diseased and nondiseased group respetively (assuming normal distribution), sensitivity and speifiity are the sensitivity and speifiity of the ontinuous marker at threshold. Therefore, for a ontinuous marker, we have PAR% = (w q ue )/w. Table 2 shows the numerial relationships among sensitivity and speifiity and PAR% of the quantifiation for various thresholds in Figure 2. Distinguishing the use of biomarkers/risk fators for aner detetion and prevention Above we presented the numerial relationships between sensitivity, speifiity and PAR% for binary and ontinuous biomarkers (Tables 1 and 2). Below we use examples to illustrate the importane of distinguishing between the use of biomarkers for aner detetion/risk predition and for aner prevention sine the onsequenes of false positive and false negative findings may differ substantially in these two ontexts. Example 1: Genotype and bladder aner. A geneti assoiation study [11] showed strong evidene that the opy number of gene GSTM1 is signifiantly assoiated with risk of bladder aner, with an OR = 1.9 orresponding to the GSTM1 null genotype (51% prevalene). If this marker were used as a binary marker for bladder aner detetion in the general population, it would result in 66% sensitivity and 50% speifiity, a poor marker for diagnosti purposes. However, if a drug were to be developed that targeted the pathway(s) by whih GSTM1 null inreases risk, and if the drug were 100% effetive in preventing bladder aner without toxi side effets (and ignoring osts), then treatment of all marker arriers would redue bladder aner by 31% (PAR%),

6 Li et al. BMC Medial Informatis and Deision Making 2014, 14:15 Page 6 of 12 Table 2 Numerial illustration for alulating RPB of hypothetial ontinuous markers using data of three aners as examples Biomarker/exposure harateristis a Ratio of population benefit (RPB) Net benefit (NB) (Threshold) b OR PAR% Sen. Spe. RPB RPB RPB NB NB NB (EA, f 0.03) (Breast a, f 0.06) (Ovarian a, f=0.15) (EA, f=0.03) (Breast a, f=0.06) (Ovarian a, f=0.15) Fixed sensitivity (95%) Fixed speifiity (95%) Balaned sensitivity & speifiity Numerial relationships between Odds ratio and PAR% of ontinuous markers and their RPB based on the data of three studies. a= aner. EA= esophageal adenoarinoma. f = loss-adjustment fator of quality-adjusted life year. NB= net benefit. OR= odds ratio. Sen.= sensitivity, Spe.= speifiity. The disease prevalene of population used for the table = As shown in the formula of RPB for ontinuous markers, disease prevalene w will diretly affet the value of RPB. a Hypothetial ontinuous biomarker/exposure with assumed distributions as desribed in Figure 1. b Threshold used for ontinuous biomarkers positive are the thresholds shown in Figure 1 (three vertial bars). Using a threshold that lead to sensitivity and speifiity losest to the upper left orner of ROC urve oordinates for utoff. whih would represent a substantial publi health benefit. One way to quantify suh a benefit an be performed using the method developed in this paper as shown in example 4. Example 2: Smoking and lung aner. Using Table 1, if the prevalene of smoking (risk fator) in a population is 30%, and the OR of smoking for lung aner risk is estimated to be 10- to 20-fold higher than the non-smokers, then the orresponding PAR% value is 73-85% (had all smokers not smoked, there would have been 73% to 85% fewer lung aners). The orresponding false positive fration is about 29.3%, whih indiates among the nonlung aner group (normal), 29.3% are smokers. This high false positive fration may be tolerable for lung aner prevention sine reduing 73-85% of lung aners at the expense of abstaining from smoking is likely aeptable. (If other diseases aused by smoking are onsidered, this argument is even stronger). Quantifiation of suh benefit an be aomplished using the method developed in this paper as shown in example 3. Quantitative evaluation of the benefit of using biomarkers for disease detetion/predition and disease prevention at the population level The above numerial analyses and speifi examples indiate that traditional measures of assoiation (OR, PAR%, sensitivity, speifiity, and others) an have dramatially different impliations depending on whether they are applied to risk predition, early detetion or prevention of disease. Sine false positive and false negative lassifiations are unavoidableinpratie[12],weproposeanindex,the naïve ratio of population benefit (RPB), whih takes into aount the adverse effets of mislassifiation, for evaluating the impat of using biomarkers for early detetion/risk predition and preventive interventions on a disease at the population level. Unlike OR, PAR%, sensitivity, speifiity, and

7 Li et al. BMC Medial Informatis and Deision Making 2014, 14:15 Page 7 of 12 Figure 2 Disease predition performane evaluated by ROC urves for the hypothetial ontinuous markers with different relative risks. ROC urves for ontinuous risk marker with different odds ratios (from bottom to top OR = 1.5, 2, 4, 10, 20, 50), whih orresponding to the distribution plots of ontinuous markers shown in Figure 1. The rosses orrespond to a fixed sensitivities; the irles to a fixed speifiities; and triangles to equal sensitivities and speifiities. Their orresponding PAR% values are shown in Table 2. other similar measures whih do not diretly depend on disease prevalene, the RPB does aount for disease prevalene in a reasonable way as shown in the following parts of the paper. This new index is not intended for evaluating or omparing the predition auray of biomarkers or predition models; instead it is intended for analyzing the potential benefit for a population using a previously-seleted biomarker for disease intervention after taking into aount potential adverse effets. RPB for binary markers/risk fators Using the 2 2 ontingeny table introdued earlier, if no biomarker is used for early aner detetion/aner risk predition, lethal aner ases will our (a + ); with a subset of individuals (b + d ) remaining aner free. The quantifiation of lives lost in this situation is f 1 (a + ), and lives gained is f 2 (b + d), with the negative sign indiating loss; ( f 1 represents naïve quantifiation of lives lost due to aner ases disovered at a late inurable stage), a positive value, f 1 represents lives gained if aner is deteted early, and f 2 represents naïve quantifiation of lives gained due to non-aner subjets who are not lassified as aner (gain due to perfet markers with no false positives and f 2 represents loss due to a false positive all). If a binary biomarker is used for aner detetion, then let a be the aner ases that will be deteted earlier (true positive), and b be the number of non-aner ases are lassified as aner due to false positives assoiated with this biomarker. The sum of gains and losses assoiated with this biomarker is f 1 a + ( f 1 )+( f 2 b)+f 2 d. Note, the sum of losses and gains assoiated with not using the biomarker f 1 (a + )+f 2 (b + d); hene, the hange of total net gain for omparison using a biomarker vs. no biomarker is f 1 a f 2 b, (no hange for f 1 + f 2 d when omparing the two sums, assuming false negative alls will be treated the same as no biomarker). For a population, if all aner ases ould be deteted early without false positives (an ideal marker), the sum of gains and losses for the population is f 1 (a + )+f 2 (b + d). Therefore, assuming binary biomarkers for aner detetion are not perfet (with false positives and false negatives), a naïve estimation of the ratio of population benefit (RPB) an be estimated by RPB ¼ 1 a f 2 b ¼ af 1 bf 2 f. f 1 ðaþþþf 2 ðbþdþ af 1 þbf 2 þf 1 þdf 2 Note that hanges in disease prevalene are aounted for in the RPB alulation, whih uses all of the terms defining prevalene (a+)/(a+b++d). The RPB is different from Net Benefit (NB) based on deision urve analysis [13,14]. NB = (a wb)/(a+b++d), where w is the weight for ounting the ost of false positive relatives to the ost of false negatives. The denominator of NB ounts the ost of the overall population, whereas the denominator of RPB only ounts the ost for worst possible performane of a marker i.e. the ounts for true positive a and true negative d both are 0 in predition. In addition, RPB also onsiders adverse effet (δ) due to intervention as shown in the next setion. Therefore, RPB is more sensitive in evaluating false positive or false negative osts ompare to NB. The adjusted RPB for potential onfounders an also be obtained by the weighted average of individual RPB for eah strata in stratified analysis, a similar idea to the adjusted PAR% mentioned above [9]. RPB adj ¼ X a p i f 1i b i f 2i i, a i f i 1i þb i f 2i þ i f 1i þd i f 2i where p i is the proportion of ases in stratum i, anda i, b i, i, d i, f 1i,andf 2i are same as in RPB above but are estimated from speifi stratum i. RPB is a perentage of net gain, whih is a ratio of net gains in the group of marker arriers (or risk fator exposed), inluding diseased and non-diseased, against overall gain estimated by quantifying the losses and gains due to false positive, false negative, true positive and true negative. Disease prevalene is onsidered in RPB alulation. For aner prevention with onsideration of adverse effets (i.e. prevention measure is applied to the arriers of a preditive (or risk-ausal) biomarker or those exposed to a risk fator), the RPB ¼ aηf 1 aδ bδ, af 1 þbf 2 þf 1 þdf 2 similarly, f 1 and f 2 are defined as the same as the binary marker mentioned above; η represents the effiay of a prevention measure (i.e. the perentage of aner redued due to a prevention measure); and δ represents

8 Li et al. BMC Medial Informatis and Deision Making 2014, 14:15 Page 8 of 12 possible adverse effet of a prevention measure (i.e. side effet of a drug for aner prevention). When there is no adverse effet from a prevention measure, δ = 0. Numerially, RPB ould be negative, 0 or positive, whih indiates detrimental, neutral, or benefiial overall effets at the population level, respetively. In addition, the absolute gain for early aner detetion or aner prevention may be quantified as (af 1 -bf 2 ) h(a+b), and (aηf 1 -aδ -bδ) h(a+b) respetively, where h is the oeffiient for the ost of prevention or treatment of exposed subjets or subjets positive for speifi markers. Following are two examples illustrating the use of RPB for binary markers and risk fators. Example 3: Smoking and lung aner. Sine there are no negative health effets due to abstaining from smoking, we set δ = 0, and RPB beomes aηf 1 /(af 1 +bf 2 +f 1 + df 2 ), where η in the example represents effiay (%) of lung aner redution due to abstaining from smoking. Example 4: Genotype and bladder aner. GSTM1isa bladder aner assoiated biomarker (marker prevalene = 51%, OR = 1.9). If a drug were developed that targeted the effet assoiated with the null GSTM1 variant, and if all arriers of the risk variant were treated with the drug, the drug had no adverse side effets and is 100% effetive (δ =0, η =1), then RPB = af 1 /(af 1 +bf 2 +f 1 +df 2 ). However, if the effiay of a drug η is muh less than 1 and/or the drug has adverse effets (δ > 0), then RPB will be smaller or even ould beome negative. In pratie, properly quantifying f 1 and f 2 may be very omplex. However, if the analyses of the effiay of the drug for aner prevention and early detetion are only limited to the diseased group (ignore b and d and f 1 0) then RPB is equal to the marker s sensitivity multiplied by ðηf 1 δþ f 1. Example 5: CNV and neuroblastoma. A opy number variation assoiated with neuroblastoma was reported reently [15]. The prevalene of the marker (1q21.1) in the general population is about 9%, and the OR of the marker (opy loss) for neuroblastoma risk is estimated to be around 3. If this marker were dihotomized as a binary marker for prediting the absene or presene of the disease, it will result in a 23% sensitivity and 91% speifiity, with a PAR% of approximately 15%, whih indiates the marker ould aount for about 15% of neuroblastoma risk if the disease is truly aused by the CNV (opy-number variation). Assume a drug is developed that targeted this marker (1q21.1) for prevention. If the drug is 100% effetive in disease prevention and had no side effets and all persons who were arriers for the marker were treated with the drug, it would redue the total disease ases by 15% (PAR%). However, in the more likely senario, drugs have signifiant side effets and are not 100% effetive suh that more extensive risk benefit analyses are needed. The RPB proposed in this paper ould be used for quantifying and evaluating the feasibility for population intervention in suh a ase. Example 6: RPB alulation for three aners for binary markers or risk fators. The utility weights for quality-adjusted life years have been estimated for surgial treatment of esophageal adenoarinoma [16], breast aner [17], and ovarian aner [18]; these are 0.97; 0.94; and 0.85 respetively. The orresponding adjustment fators for loss of quality of life (f 2 ) are 0.03, 0.06, and 0.15 respetively for the three aners. If a aner were deteted early and intervention were a omplete suess, this would lead to a benefit value of 1 (true positive deteted early); if a subjet were wrongly diagnosed with aner and surgery was done, the ost value an be represented as the loss adjustment fator for quality-adjusted life year. This leads us to have f 1 =1,f 2 = loss-adjustment fator of a disease intervention to alulate RPB proposed above. Using breast aner as an example, f 2 = 0.06, sine RPB ¼ af 1 bf 2 af 1 þbf 2 þf 1 þdf ¼ 2 a 1 b 0:06 a 1þb 0:06þ 1þd 0:06,wherea, b,, and d are the number of true positive, false positive, false negative, and true negative due to using a biomarker for disease outome predition for intervention. In many ases, the OR has been estimated for a marker or risk fators. In Table 1, we show the relationship among RPB and various ORs and prevalene of a marker or risk fator using the loss-adjustment fators of the three aners as examples. We also alulated net benefit (NB) values under these senarios for omparison. For instane, from Table 1, if a biomarker has 1% prevalene with OR 10 for breast aner risk, then the RPB = 0.004, if OR = 20, RPB = If a biomarker has 10% prevalene with OR 10, then the RPB = ; if OR = 20, RPB = It will be possible to apply these priniples to other diseases, novel risk assessments and new treatments as additional data beome available. Table 1 shows numerial examples with the assumption of 1% disease prevalene. Disease prevalene of a population will diretly affet RPB as the a and in the 2 2 table are used in the alulation of RPB. For instane, if disease prevalene is hanged from 1% to 3%, and assuming a biomarker (or exposure) prevaleneof30%,therpbforariskbiomarkerwithvarious OR values 1.5, 2, 4, 10, 20, and 50 will be: 0.05, 0.086, 0.172, 0.266, 0.311, respetively for EA; , -0.04, 0.02, 0.084, 0.116, respetively for breast aner; and -0.18, , , , , respetively for ovarian aner. RPB for ontinuous markers and risk fators For a ontinuous marker, the naïve ratio of population benefit for early aner detetion (present or absent) or prevention an be alulated as RPB ¼ R R wf f 1 dðþdx x ð 1 w Þf 2 f d ðþdx x R R wf f 1 dðþdxþ x ð 1 w Þf R 2 f R d ðþdxþwf x f 1 dðþdxþ x ð 1 w Þf 2 f d ðþdx x where w is disease prevalene in a population, f d (x) and f d ðþ x are the probability density distribution of a,

9 Li et al. BMC Medial Informatis and Deision Making 2014, 14:15 Page 9 of 12 biomarker in the diseased and the non-diseased group respetively and is the utoff threshold for positive and negative alls for a ontinuous marker. This formula an also be modified to address potential onfounding fators using stratified analysis. The adjusted RPB for a onfounding fator of a ontinuous marker is: RPBadj ¼ P p i i R R w i f f 1i d iðþdx x ð1 w i Þf f 2i di ðþdx x R R w i f f R 1i diðþdxþ x ð1 w i Þf f R, 2i di ðþdxþw x i f f 1i diðþdxþ x ð1 w i Þf f 2i di ðþdx x where p i is the proportion of ases in stratum i, andthe quantifiation for loss f 1i,andf 2i are the same as in RPB above but are estimated in speifi stratum i. The density distribution funtions f di (x) and f di ðþ x are the density distributions of a biomarker in the disease and the non-disease group respetively in stratum i.similar to binary markers, the naïve ratio of population benefit for aner prevention using ontinuous markers with onsideration of adverse effets ould be alulated as RPB ¼ R R wf ηðþf x 1 d ðþdx w x R δðþf x dðþdx x ð 1 w Þ δðþf x d ðþdx x R R wf f 1 dðþdx x þ ð 1 w Þf R 2 f d ðþdx x þ wf R, 1 f dðþdx x þ ð 1 w Þf 2 f d ðþdx x where η(x) is effiay (as a funtion of x) of a prevention measure and δ(x) represents adverse effet due to a prevention measure. Similar alulations an be used to obtain RPB if δ(x) and η(x) are dependent on f d (x) andf d ðþ. x The total lives gained for early aner detetion and aner prevention using a ontinuous marker may be quantified as R wf 1 f d ðþdx x ð1 wþf 2R f d ðþdx x R hw f d ðþdx x þ ð1 wþ R f d ðþdxþ, x and wf 1R ηðþf x d ðþdx wr x δ ðþf x d ðþdx x ð1 wþ R δðþf x d ðþdxþ h x wr f d ðþdxþ x ð1 wþ R f d ðþdxþ x, respetively. As defined for binary markers above, h is the oeffiient for quantifiation of the ost of prevention or treatment for exposed subjets or subjets with positive markers; the other parameters remain the same as defined for binary markers. For example, BMI (body mass index) is a ontinuous marker [19]; studies have shown the assoiation between high BMI and esophageal adenoarinoma risk [20-22]. If this is a ausal assoiation, then BMI may not be a robust marker for deteting presene or absene of disease. However, if BMI were onsidered as a modifiable risk fator, then false positives may be tolerable sine reduing BMI for those people who would never develop esophageal adenoarinoma had their BMI not been redued will likely not have substantial detrimental effets (δ = 0, RPB always >0). If we use BMI 30 as the threshold and assumed no negative effet (δ =0) for reduing BMI for those who have a BMI 30, then the RPB ¼ R wf ηðþf x 1 d R ðþdx x R wf f 1 dðþdxþ x ð 1 w Þf R 2 f d ðþdxþwf x R 1 f dðþdxþ x ð 1 w Þf 2 f d ðþdx x where η(x) is the effiay of aner redution by reduing BMI, f d (x) andf d ðþare x the BMI distribution in a speifi at risk population and low risk population, respetively. However, if negative effets our when reduing BMI ; (δ > 0), e.g., if a mediation used for weight loss is assoiated with signifiant side effets, then the RPB will be smaller. The RPB, therefore, ould be used to quantify the potential overall benefit of a prevention measure targeted to a marker or risk fator. Similar to binary markers, if the analysis of effets for prevention and disease detetion is restrited to the diseased group only (ignore f d ðþin x RPB above) for a ontinuous marker in prevention, and assuming no negative effet (δ = 0), then the RPB beomes R ηðþf x d R ðþdx x R f d ðþdxþ x f d x ðþdx, whih is an estimation of prevention effets with onsideration of at risk subjets only, and it will always have a positive value (benefit). Example 7: RPB alulation for ontinuous markers or risk fators. To evaluate the benefit of using a ontinuous biomarker or risk fator for disease intervention, the probability density distribution of the marker in the population of disease outome f d (x) and non-disease outome f d ðþan x be estimated from observed population data. Then, to alulate RPB of the ontinuous biomarker, a threshold of the biomarker is hosen (i.e., any subjets with the biomarker level above the threshold will be predited to have the disease outome); then numerial integration an be used to obtain RPB for the ontinuous markers using the formula above. For example, using the same loss of quality of life adjustment fators of the three aners shown in Example 6 (the loss fator f 2 are 0.03, 0.06, and 0.15 for the three aners), assume the probability distributions of a ontinuous marker in the population with disease f d (x) andwithoutdiseasef d ðþfollow x the normal distributions with different means as shown in Figure 1. Using numerial integration for the RPB formula above, we alulated the RPB for various senarios in whih the hypothetial ontinuous markers have different OR for the aner risk as shown in Table 2. In these alulations, the thresholds of the ontinuous biomarker were hosen to demonstrate various possibilities for outome predition inluding fixed sensitivity, fixed speifiity, and balaned sensitivity and speifiity. Table 2 shows numerial examples with assumption of 1% disease prevalene for RPB alulation. Disease prevalene w of a population will diretly affet RPB value. For example, for a breast aner biomarker with balaned sensitivity and speifiity if the diseaseprevaleneishangedfrom1%to3%,theorresponding RPB values for a risk biomarker with variousorvalues1.5,2,4,10,20,and50willbehanged from , ,-0.222, , , for 1% prevalene to -0.12, -0.09, , 0.058, 0.112, respetively for 3% prevalene. Disussion With rapid advanes in various tehnologies, a large numbers of biomarkers have been reported to be assoiated

10 Li et al. BMC Medial Informatis and Deision Making 2014, 14:15 Page 10 of 12 with various diseases, inluding aner. Translating those to the lini and for publi health benefit is a ritial but diffiult next step. For example, genome-wide assoiation studies are identifying hundreds of SNPs assoiated with a variety of diseases. While the rih disoveries from suh studies ontinue to prompt investigation of pathwaytargeted interventions for disease prevention and therapy [6], it is generally believed that use of single or ombined SNP information an ahieve only modest improvement in disease risk predition or early detetion programs for individuals in the general population as ompared to urrent linial sreening modalities [23-27]. This undersores the need for quantitative evaluation of both the risk and potential benefit of biomarkers sine in this proess many fators need to be onsidered inluding sensitivity and speifiity of the marker used for risk predition or targeted therapy, disease prevalene and quantitative relationship between biomarker levels and meaningful risk measures, ost, and risk/benefit analyses [28-30]. In this paper, we all attention to the need to distinguish and quantify the onsequenes of false positive and false negative diagnoses of a marker for prevention and early detetion/risk predition. The RPB estimation an be onfounded due to the marker or risk exposure. Adjustment for potential onfounders in estimating RPB need be onsidered. Table 1 shows the numerial relationships between measurements ommonly used for aner detetion and prevention. For a marker with a low prevalene, the sensitivity of the marker is very low even when it has a high OR value. Combining multiple SNP/CNV markers with low OR values for predition may have limited effets at the population level, i.e. a person who arries all 10 risk SNPs will be at high risk for the disease; however very few people arry all 10 SNPs in the general population, thus leading to a low PAR% value. Therefore, using suh a panel may have a low impat on disease detetion or risk predition in the general population, although it might be applied to the few individuals in that ategory. Based on three published data sets, the numerial alulations of RPB for the three aners are presented in Tables 1 and 2. The alulations show that due to low speifiity of a marker and the disease intervention ation based on the predition of the markers in the tables, risk is larger than benefit (RPB < 0) in many ases. However, we used the loss of quality of life adjustment fator from the three aners in a simple way to demonstrate the appliation of the RPB. The risk and benefit may be affeted by many fators suh as age, time, or unknown onfounders. Six phases are reommended for developing effetive biomarkers, with longitudinal studies onsidered essential for validation [31]. Pepe et al. [32] presented a omprehensive method for evaluating the preditiveness of a biomarker and its performane as a disease lassifier. The appliation of PAR% was evaluated for benefit of ommunity-based efforts to prevent disease using a speifi aner marker by Waholder [33]. Furthermore, the quantitative onnetion between biomarker levels in ases and ontrols and linial meaningful risk measures or testing also has been arefully evaluated by Wentzensen and Waholder [30], adding a useful tool for apprising andidate biomarkers at an early stage. The RPB proposed in this study takes aount of both auray of outome predition of a marker and benefit for a population if the marker were used for an intervention. The predition auray of biomarker(s) should be assessed and ompared with validated or well-established tools (suh as area under the urve, integrated disrimination improvement, net relassifiation improvement et.). Then the value of biomarker(s) should be further evaluated for risk and benefit if it were to be applied in a large population for intervention. In this paper, we assumed the seletion of biomarker(s) for predition has been ompleted, andweproposetherpbforriskandbenefitanalysisatthe population level when a given marker or risk fator is used for disease intervention. Speifially, we onentrate on the framework of risk/benefit analysis in using a marker for disease prevention and detetion/predition. The 1,000 Genomes Projet is expeted to disover substantially more SNP markers and other variants that have frequeny between 0.5 to 5%. Those data ould be analyzed by the methods presented in this paper. Thus far, the performane of SNP/CNV for disease risk predition or risk stratifiation still needs improvement [34], while their potential for disease prevention or targeted therapy [7] and predition of prognosis [35] is substantially enouraging. A broad risk/benefit analysis will be needed when translating the results of suh studies into linial use for aner risk predition, detetion and prevention. Greenland [36] pointed out that the evaluation of marker predition models are linked to preditor-onditional performane, ut-point hoies, and error osts; and there is a need for reorientation toward ost-effetive predition. We extended the issue further by distinguishing between using biomarkers for early disease detetion and using them for prevention. The feasibility or value of using biomarkers in the two senarios ould be generalized by risk/benefit analysis, a researh diretion that has been proposed in previous studies [37,38]. The proposed RBP for binary and ontinuous markers/exposures an be extended to health eonomis studies. The National Caner Institute reently identified hallenges for aner epidemiology in the 21st entury [39]. Eight overarhing reommendations with orresponding ations were proposed by the sientifi ommunity for onsideration [40]. Here, we propose methods that an be used for assessing risk and benefit of disease intervention based on a biomarkers or risk fators, whih are

11 Li et al. BMC Medial Informatis and Deision Making 2014, 14:15 Page 11 of 12 partiularly pertinent to two of the eight reommendations: (1) balane the epidemiology researh portfolio beyond traditional emphasis on disovery and etiology researh to enompass development and evaluation of linial and population interventions, implementation, dissemination, and outomes researh ; and (2) support knowledge integration and meta researh (systemati reviews, modeling, deision analysis et.) to identify gaps, inform funding, and to integrate epidemiologi knowledge into deision making. The RPB is intended to illustrate an approah to assess the risk and potential benefits using a marker/risk fator for intervention in both early detetion and prevention. As suh, it is a general framework, and requires a proper estimation of risk/benefit quantifiations (f 1 and f 2 ) in the RPB model for eah disease. Substantial effort may be needed to properly estimate suh parameters for a biomarker to properly evaluate the feasibility of using the biomarkers for different senarios suh as early detetion, risk predition, and prevention. Conlusions Making use of the disovered biomarkers/risk fators from epidemiologial and linial researh for linial deision making is a key step to translate the disoveries into pratial use to ahieve health benefits. Risk benefit analysis provides ruial information for disease intervention deision making. It is worthwhile to distinguish and quantify the differenes between the use of biomarkers/risk fators to identify preventive interventions versus their use in disease risk predition and early detetion. The RPB proposed in this paper not only onsiders the absolute risk of a disease in intervention, but also takes into aount risk-benefit effets simultaneously for a marker/exposure at the population level. Using onrete examples, we demonstrate that RPB developed in this study is a useful tool for quantitatively assessing the risk and benefits in using a biomarker/risk fator for intervention in both early detetion and prevention. Abbreviations CNV: (somati) Copy number variation; OR: Odds ratio; PAR%: Population attributable risk perent; RPB: The naïve Ratio of population benefit; SNP: Single nuleotide polymorphism. Competing interests All authors of this paper delare that they have no ompeting interests. Authors' ontributions Author XL initiated the prototype of the manusript in the environment of lose working relationships with other o-authors of the manusript. PLB and BJR provided input from the linial appliation point of view. TLV provided input from the epidemiology and population siene point of view. All authors were involved in vigorous disussions during the writing of the manusript. XL performed the omputer programming for all numerial alulations. All authors of the manusript were atively involved in the editing and finalization of the submitted manusript. All authors read and approved the final manusript. Aknowledgements This work was supported by the National Caner Institute at the National Institutes of Health (Grants P01 CA091955, RC1 CA146973, K05 CA124911, R01 CA and R01 CA179949). We thank Dr. Dennis Chao and Dr. Ying Q Chen of Fred Huthinson Caner Researh Center for reading and giving omments on the near final version of the manusript. Author details 1 Divisions of Publi Health Sienes, Fred Huthinson Caner Researh Center, 1100 Fairview Ave. N., Seattle, WA 98109, USA. 2 Human Biology, Fred Huthinson Caner Researh Center, 1100 Fairview Ave. N., Seattle, WA 98109, USA. 3 Department of Mediine, University of Washington, 1959 NE Paifi Street, Seattle, WA 98195, USA. 4 Department of Epidemiology, University of Washington, 1959 NE Paifi Street, Seattle, WA 98195, USA. 5 Department of Genome Sienes, University of Washington, 1959 NE Paifi Street, Seattle, WA 98195, USA. Reeived: 17 September 2013 Aepted: 18 February 2014 Published: 6 Marh 2014 Referenes 1. Pepe MS, Janes H, Longton G, Leisenring W, Newomb P: Limitations of the odds ratio in gauging the performane of a diagnosti, prognosti, or sreening marker. Am J Epidemiol 2004, 9: Jakobsdottir J, Gorin MB, Conley YP, Ferrell RE, Weeks DE: Interpretation of geneti assoiation studies: markers with repliated highly signifiant odds ratios may be poor lassifiers. PLoS Genet 2009, 2:e Boyko EJ, Alderman BW: The use of risk fators in medial diagnosis: opportunities and autions. J Clin Epidemiol 1990, 9: Roulston JE: Assessment of preditive values of tumor markers. Methods Mol Med 2004, 97: Kattan MW: Judging new markers by their ability to improve preditive auray. J Natl Caner Inst 2003, 9: Hirshhorn JN: Genomewide assoiation studies illuminating biologi pathways. N Engl J Med 2009, 17: Marian AJ: Moleular geneti studies of omplex phenotypes. Transl Res 2012, 2: Markush RE: Levin's attributable risk statisti for analyti studies and vital statistis. Am J Epidemiol 1977, 5: Rothman KJ, Greenland S, Lash TL: Modern Epidemiologgy. Philadelphia: Lippinott, Williams & Wilkins; Pepe MS: The Statistial Evaluation of Medial Tests for Classifiation and Predition. New York: Oxford University Press; Garia-Closas M, Malats N, Silverman D, Dosemei M, Kogevinas M, Hein DW, Tardon A, Serra C, Carrato A, Garia-Closas R, Lloreta J, Castano-Vinyals G, Yeager M, Welh R, Chanok S, Chatterjee N, Waholder S, Samani C, Tora M, Fernandez F, Real FX, Rothman N: NAT2 slow aetylation, GSTM1 null genotype, and risk of bladder aner: results from the Spanish Bladder Caner Study and meta-analyses. Lanet 2005, 9486: Li X, Blount PL, Vaughan TL, Reid BJ: Appliation of biomarkers in aner risk management: evaluation from stohasti lonal evolutionary and dynami system optimization points of view. PLoS Comput Biol 2011, 2:e Steyerberg EW: Clinial Predition Models: A Pratial Approah to Development, Validation, and Updating. New York: Springer; Steyerberg EW, Vikers AJ, Cook NR, Gerds T, Gonen M, Obuhowski N, Penina MJ, Kattan MW: Assessing the performane of predition models: a framework for traditional and novel measures. Epidemiology 2010, 1: Diskin SJ, Hou C, Glessner JT, Attiyeh EF, Laudenslager M, Bosse K, Cole K, Mosse YP, Wood A, Lynh JE, Peor K, Diamond M, Winter C, Wang K, Kim C, Geiger EA, MGrady PW, Blakemore AI, London WB, Shaikh TH, Bradfield J, Grant SF, Li H, Devoto M, Rappaport ER, Hakonarson H, Maris JM: Copy number variation at 1q21.1 assoiated with neuroblastoma. Nature 2009, 7249: Hur C, Nishioka NS, Gazelle GS: Cost-effetiveness of aspirin hemoprevention for Barrett's esophagus. JNatlCanerInst2004, 4: