Evaluating Predictive Modeling s Potential to Improve Teleretinal Screening Participation in Urban Safety Net Clinics

Size: px

Start display at page:

Download "Evaluating Predictive Modeling s Potential to Improve Teleretinal Screening Participation in Urban Safety Net Clinics"

Felicia Baker
5 years ago
Views:

1 Evaluating Predictive Modeling s Potential to Improve Teleretinal Screening Participation in Urban Safety Net Clinics Omolola Ogunyemi, PhD 1,2, Senait Teklehaimanot, MPH, 2 Lauren Patty Daskivich, MD, 3 Erin Moran, MA, 1,2 Sheba George, PhD 1,2 1 Center for Biomedical Informatics 2 Charles Drew University of Medicine and Science, Los Angeles, California 3 Los Angeles County Department of Health Services

2 Overview Background Diabetic retinopathy Pre-telemedicine screening process for diabetic retinopathy in inner city Los Angeles Study goals Methods Results Discussion

3 Background Diabetic retinopathy: Damage to blood vessels of the retina caused by diabetes Left untreated can lead to blindness Leading cause of blindness in US adults aged 20 to 74 years Risk factors include poor blood glucose control high blood pressure high cholesterol smoking length of time a patient has had diabetes

4 Background On average 60% of adults across the US with diabetes get annual screening in accordance with American Diabetes Association (ADA) guidelines Less than 25% of inner-city/safety net diabetic patients in the US get annual screening in accordance with ADA guidelines Previously conducted a study on feasibility of using telemedicine to improve screening rate for diabetic retinopathy in US urban safety-net clinics Over 2700 diabetic patients from 6 clinics screened for diabetic retinopathy using teleretinal screening Safety net screening rate of 29% achieved in prior study Many inner city safety net clinics lack the resources to integrate teleretinal screening into primary care visit

Background Process for retinopathy screening in South LA safety net setting in the absence of telemedicine Federally Qualified Health Center Diabetic patient seen at primary care clinic LA

5 Background Process for retinopathy screening in South LA safety net setting in the absence of telemedicine Federally Qualified Health Center Diabetic patient seen at primary care clinic LA County DHS Hospital Patient referred to county health facility for routine retinopathy screening 4 8 months after referral Patient screened at county health facility, returns if treatment necessary

6 Study Goals Work towards increasing the effectiveness of teleretinal screening in resource-poor safety net settings by: Developing predictive models for diabetic retinopathy based on routinely available clinical data from patients medical records (current presentation) Developing tools that help clinics perform targeted outreach to diabetic patients who are most at risk for retinopathy (future goal)

7 Methods IRB approval for study obtained from CDU IRB Collected medical records data via retrospective medical records review for 513 diabetic patients who Were seen at any of six South Los Angeles clinics participating in CDU teleretinal screening study in 2011 Had a teleretinal screening performed with a retinopathy determination provided by a reader (board certified ophthalmologist) in 2011 Patients screened for Mild non-proliferative diabetic retinopathy (NPDR) Moderate NPDR Severe NPDR Proliferative diabetic retinopathy (PDR) Clinically significant macular edema Other conditions (glaucoma, cataracts, non-diabetic maculopathy, etc.)

8 Methods Routinely available clinical variables investigated for their predictive value for diabetic retinopathy Outcome: Diabetic retinopathy present All variables considered (23) Age Ethnicity/race Education Insulin dependence Number of years patient has had diabetes Gender Marital Status Household income Insurance Body mass index Hemoglobin A1C value Peripheral vascular disease Hypertension Nephropathy Depression Dyslipidemia Other (hypothyroidism, etc.) Cerebrovascular accident/stroke Other heart-related diagnosis Neuropathy Erectile dysfunction Obesity Previous diagnosis & treatment of retinopathy

9 Methods Since different classification methods exhibit different biases, first learned Bayesian network classifiers Radial basis function (neural) network classifiers on all 23 clinical variables available for 513 cases. Used 10-fold crossvalidation. Next, performed feature subset selection using depth-first search to establish a useful subset of risk factors/clinical variables Learned Bayesian network and radial basis function network classifiers on subset. Used 10-fold cross validation. Measured sensitivity, specificity, accuracy and Area Under the ROC Curve (AUC) for all classifiers

10 Results Classification results (all 23 available clinical variables used): Bayesian Radial basis network function network Correctly classified cases Incorrectly classified cases Accuracy 76.6% 71.5% Sensitivity 28.5% 17% Specificity 93% 90% Positive predictive value 57.8% 36.7% Negative predictive value 79.3% 76.2% Area Under the ROC Curve

11 Feature subset selection: Results Subset of variables utilized following depth-first search feature subset selection (6) Outcome: Diabetic retinopathy present Hemoglobin A1C value Number of years patient has had diabetes Hypertension Marital Status Neuropathy Previous diagnosis & treatment of retinopathy

12 Results Classification results (subset of 6 clinical variables used): Bayesian Radial basis network function network Correctly classified cases Incorrectly classified cases Accuracy 77.2% 75% Sensitivity 26.2% 18.5% Specificity 94.5% 94.3% Positive predictive value 61.8% 52.2% Negative predictive value 79% 77.3% Area Under the ROC Curve

13 All classification results: Bayesian network (all variables) Results Bayesian network(subs et of variables) Radial basis function network (all variables) Correctly classified cases Incorrectly classified cases Accuracy 76.6% 77.2% 71.5% 75% Radial basis function network (subset of variables) Sensitivity 28.5% 26.2% 17% 18.5% Specificity 93% 94.5% 90% 94.3% Positive predictive value 57.8% 61.8% 36.7% 52.2% Negative predictive value Area Under the ROC Curve 79.3% 79% 76.2% 77.3%

14 Discussion Bayesian network classifiers had better sensitivity, specificity, accuracy, positive predictive values and AUCs on this particular dataset Positive predictive value of classifiers learned on feature subset better than those learned on full set of available clinical variables Clinics currently lack resources to integrate image taking for teleretinal screening into primary care visit Identifying patients most at risk of developing retinopathy through predictive models would be helpful for clinics Results shows the promise of predictive modeling approach but more work needs to be done to improve predictive models

15 Acknowledgments 6 South LA FQHCs Rao Ilapakurthi, MS Lauren Patty Daskivich, MD Elizabeth Terrien, MA Miguel Unzueta, MD Otaren Aimiuwu, MPH, Michelle Banks, MD Kevin Lopez Sheba George, PhD Richard Baker, MD Allison Fish, JD, PhD David Martins, MD James Smith, MD Keith Norris, MD Alicia Eccles, MPH Erin Moran, MS Senait Teklehaimanot, MPH Angela Nossett, MD (MLK MACC, Los Angeles County DHS) Jorge Cuadros, OD, PhD (UC Berkeley/EyePACS)

16 Acknowledgments This work was supported in part by the US National Institutes of Health under grant U54 MD S2 (formerly U54 RR S2): Accelerating Excellence in Translational Science (AXIS). AXIS webpage:

17 Thank you! Questions?

Teleretinal Screening for Diabetic Retinopathy: A Novel Approach to Reduce Screening Burden on the Healthcare Systems within Central Texas

Teleretinal Screening for Diabetic Retinopathy: A Novel Approach to Reduce Screening Burden on the Healthcare Systems within Central Texas Robert W. Wong, MD; Maria Benson, BS; Jose A. Martinez, MD; Mark