A workshop on making sense of research results and statistics to improve underwriting. AAIM 2015 Mike Fulks

Transcription

1 A workshop on making sense of research results and statistics to improve underwriting AAIM 2015 Mike Fulks

2 Industry rating approaches Table rating (relative risk) Multiple of expected risk (which is the risk for all standard and preferred insureds together). Expressed as debits (often units of +25%)such as 50 (or total risk of 150% or 2 tables) or as credits such as 25. will be today s focus using laboratory test and physical measurements research Flat extra (absolute extra risk) Increase in risk expressed in $/$1,000 of face amount extra deaths/1,000 lives/yr. Used when extra risk in deaths is the same regardless of expected mortality based on age, sex or table rating. cancer- temporary aviation-permanent 2

3 150% of what? Of Expected (all-cause) Mortality for insured lives. 100% likely based on a recent VBT (valuation basic table) representing fully-underwritten industry experience for preferred and standard combined, With adjustment by your pricing actuary for your market and und. approach factoring in anticipated mortality improvements. How well does the researchreference pool match the standard insurance applicant? Does the relative or extra risk shown in a research paper translate into similar (or higher or lower) extra risk: When applied to an applicant ref. pool which may be healthier? When applied to a diseased pool which may be healthier? 3

4 Distribution of applicants Usually 85 to 95% will be standard or preferred depending on age and market (may be lower for some markets). Distribution of offers across preferred classes? Know your product targets. To be equal across age and sex? When changing the ratings for any finding, the distributions across preferred classes, standard and substandard must be recognized and probably maintained. Shift roughly same # up as down; no credit without a debit. 4

5 Statistical evaluation There are 3 kinds of lies: lies, damned lies and statistics. Disraeli Statistics can help us make use of real world data to draw meaningful conclusions. They can be used to determine if results seen likely reflect TRUTH (or just confuse the reader into believing something not true). The % of published conclusions that are accurate has been questioned. Use to the minimum extent needed. Beware conclusions which can only be reached by statistics that you can t begin to understand or articles dominated by statistical verbiage. 5

6 Life table analysis Often used for pools of insured lives (or similar) and a wellsuited, easily understood tool for that. Splits done by durations of follow-up after underwriting accounting for varying degree of selection. Compare actual to expected mortality often based on external mortality table such as VBT. Difficult to identify comparable reference pool for non-insur. studies. For insur. life studies, just being insured for an impairment (MIB code) may mean highly selected cases. Univariate, runs low on outcomes in each cell (30) and often requires separate expected mortality pool. Usually limited to insured lives (or similar data) and very large data-sets. 6

7 Survival by duration but univariate (ignores maldistribution of other factors). Great for comparing treatments or single findings when disease associated mortality much higher than other causes and when relative risk may vary between groups over time. Kaplan Meier Plot 7

8 ROC Curve A simple (simplistic) graphical tool to evaluate how well any test result predicts the presence (sensitivity) and absence (specificity) of a condition or the risk of an event. Generates a graph and an AUC (area under the curve) identical to the c statistic. No association has AUC of 0.5; perfect association is 1.0 (or 0.0 if the association is negative). A univariate analysis requiring care that the association is not really based on other, not-included, variables such as age and sex. 8

9 ROC- which test is better? Men age AUC Urine albumin.641 Urine p/c.658 Applicants tested at CRL , VS in

10 ROC- value of unrelated tests? Women age Applicants tested at CRL , VS in

11 ROC- Test values Women age Chol/HDL Sensitivity 1- Specificity

12 Cox regression analysis The most common analysis seen in medical journals. Uses a regression algorithm to determine how well a finding A explains results B - not cause and effect! It allows use of data the way it is available in the real world by adjusting for rather than spitting by (or ignoring): Different periods of follow-up for subjects. A range of ages and both sexes accounting for the mortality impact of those variables.. Other differences between subjects that may impact outcome such as smoking, stage, grade or laboratory test results. 12

13 Why Cox is so valuable Generates a multivariate analysis. You can keep all the (limited) data together in the analysis rather than splitting by age, sex or duration so you don t have too few outcomes (the bane of any medical researcher s existence). You can account for the impact of each variable on the outcome. 13

14 How does Cox do it? It uses regression algorithms to assign part of the risk to each variable based on how well each explains the outcome (mortality). It is done with software programs such as SPSS or SAS or Stata which are loaded with the data and then the Cox button is pushed after options chosen. Assumesrelative risk associated with any variable stable over time period considered (proportional hazard assumption) and that variables are independent. 14

15 Outcome- a hazard ratio A hazard ratio (such as 1.45) reflecting the relative risk of an outcome based on: one integer to the next averaged across all values (GGT 100 vs. 101, 102, etc. or by year of age, etc.) or between categories such as male and female or between chosen bands of values against a reference band (GGT <65 (ref.), , 101+) or (as commonly done) between % grouping such as tertiles or quintiles often with the best assigned the risk of 1. 15

16 Cox limitations Failure to include variables that may be actual causes of the risk. Not capable of accurately sorting the risk between variables that are dependent or closely related such as HbA1c and fructosamine, or ALT and AST, or ALT and HBV. Include just one test of each thing. Other approaches to deal with the interaction. Lumping subjects together by including as a covariate (age and sex) masks potentially very different results. Assumption that RR stable over time if not. 16

17 Accounting for important variables Haz. Ratio 95% CI for Haz. Ratio Lower Upper CHOLESTEROL CHOLESTEROL SEX AGE The distribution of cholesterol values (mg/dl) will vary by age and sex with the lowest values in young females who will also have the lowest mortality. Adding age and sex as variables to figure out how much of the risk is for cholesterol vs. age or sex. Applicants tested at CRL with VS in

18 Same test, different pop. & view Ingle D, ON THE RISK vol29 n.1 (March, 2013) 18

19 Impact on group vs. subgroup GGT (& age-sex) by Cox 100 Applicants aged tested at CRL, VS 2011, with all other LFTs <95 th % Debits GGT Values (U/L) 19

20 Impact on group vs. subgroup GGT (& age-sex) by Cox All F M Debits GGT Values (U/L) 20

21 Confidence intervals Along with the hazard ratio, you should see confidence intervals which mostly replace traditional p values. Typically shown are the hazard ratios at the upper and lower end of the 95% CI meaning there is a 95% chance the real risk would lie inside the interval. Width of the 95% CI-largely dependent on # of outcomes (deaths). 30 or more per cell is good. If the 95% CIs overlap, then the hazard is not demonstrated to be different at a 95% significance level. May not actually be different or May have too few deaths and too wide a CI to tell. 21

22 HbA1c study- an example of missing variables, confidence intervals &- Khaw. HbA1c and mortality in the Norfolk study. Ann Intern Med 2004 HbA1c values <5%, 5-5.4%, %, 6-6.4%, etc. Had the following hazard ratios (MRs): < 5% 5-5.4% % 6-6.4% Men Women

23 HbA1c study contd. Missing variables, confidence intervals &- < 5% 5-5.4% % 6-6.4% Men ( ) 1.6 ( ) 1.8 ( ) Women ( ) 1.3 ( ) 1.6 ( ) Wide overlapping CIs (based on deaths/cell). plausible but limited support. Included only age as covariate with sex split in their main result table. differences in smoking ( %) and CV risk factors included only in a secondary analysis. 23

24 Assessing an article Did the (Cox) multivariate analysis: Compare meaningful groups or just extreme quintiles) designed to obtain a positive publishable result? Have CIs that support the conclusion? Include all important variables or at least discuss the issue? Include collinear variables (travel together) without discussion? Did the ROC curve ignore an important variable that may also be associated with both high and low results and the outcome of concern? Are the results biologically plausible and meaningful regardless of analysis and statistical methodology? 24

25 Translating research to ratings Associating low albumin with mortality What is the distribution of albumin results? Are there differences by age-sex? = Distribution When does risk go up? Does it vary by age-sex? = Alert value When does risk go up when other findings (only those which would be actually rated ) are accounted for? = Relative Risk or Underwriting action point CRL tested applicants split by sex and age 60 using Cox including age and smoking as covariates- AHOU 2015 and in press based on applicants tested 91 to 2007 at CRL with VS in

26 Distribution of lowalbumin Females Males Albumin g/dl <= % 0.3% 0.1% 0.2% Deaths 283 (29) >3.3 to % 0.6% 0.1% 0.3% >3.5 to % 0.9% 0.1% 0.4% >3.7 to % 5.2% 0.5% 2.7% >3.9 to % 14.5% 2.7% 9.7% >4.0 to % 11.5% 3.7% 9.4% >4.1 to % 13.6% 6.3% 12.4% > % 53.4% 86.5% 64.9% Appropriate reference pool? (not just highest or lowest quintile/decile) Sufficient # (accurate results) or % (useful) in each band? 26

27 Accounting for other variables-smoking Females Males Albumin g/dl <= % 9.0% >3.3 to % 13.1% >3.5 to % 14.8% >3.7 to % 17.8% >3.9 to % 16.9% >4.0 to % 15.7% >4.1 to % 14.8% > % 12.4% Split by sex and age band but use age and smoking as covariables. Account for other test results (low chol, etc.) by excluding cases with other abnormal. Get a speculative idea of risk by adding as covariable rather than excluding. 27

28 Relative Risk by Cox Males age 20-59, relative risk compared to albumin >4.2 All, with age and sex as covariates Include other lab score as variable Exclude those with other lab with high score MR MR MR 95% CI Albumin g/dl (Cox) (Cox) (Cox) Lower Upper <= >3.3 to >3.5 to >3.7 to >3.9 to >4.1 to >4.2 (ref)

29 Same data shown graphically Mortality ratios (Cox) for albumin, Males 20 to 59 Mortality Ratio Albumin Albumin adj for other tests Albumin w/ other tests OK >4.2 (ref) >4.1 to 4.2 >3.9 to 4.1 >3.7 to 3.9 >3.5 to 3.7 >3.3 to 3.5 <=3.3 Albumin (g/dl) 29

30 Low Albumin Rating Create alert values based on increased risk regardless of other findings (for each age-sex quadrant) taking into account distribution (can only review <10% or 5% or fewer). Combine age-sex if close enough to simplify for alert purposes. Create a rating (or other action) point when no other findings/explanation (or in conjunction with other findings). Establish relationship of study pool to standard insurance pool. If substandard results, can only impact a very small percentage. If percentage high, adjust standard cut-off downward and use as a preferred factor instead or in addition to rating. 30

31 Further Discussion or Cases 31

32 Case 1 Your boss suggests it is OK to take diabetics standard if otherwise standard. She based this on a published, well-done industry life table analysis using MIB coding with 15,292 person years of exposure and 71 deaths (expected 55 based on 2001 VBT) noting the MR was only 128% with a 95% CI from %. Your standard class accepts up to 135% risks. Is she correct? How likely from the model is the MR actually 128%? What data should I look at to tell? Is adding a pool of 128% risks OK for standard? Mortality Study of Policies, Milano AF, et al. JIM 2005;37. Is this 128% reflective of the risk of those presenting with DM who are otherwise standard? 32

33 Case 2 The table contains AUCs from a ROC curve for prediction of mortality by egfrusing Rule and CKD-EPI formulas and creatinine. What does it tell us about relative efficacy of each in predicting mortality risk and why is creatinine inverted? Females, age Males, age egfrby Rule egfr by CKD-EPI Creatinine (1/creat.) Which egfrcalc, Fulks M, et al. OTR2015;31. 33

34 Case 3 The table contains RR by Cox for globulin values in males age with age as a covariate. The 2 nd analysis restricts the group to those with <25 debits for BMI, albumin or Alk. Phos. Why are the RRs lower for the 2 nd analysis? Which value might be better for underwriting alert & for und. action? Mortality, all cases Mortality when BMI, albumin, AP <=125% Globulin MR 95% CI MR* 95% CI g/dl (Cox) Lower Upper (Cox) Lower Upper (ref) Serum globulin, Fulks M, et al. JIM 2014;44. 34