Evaluation of Real-time Outbreak and Disease Surveillance (RODS) system in Taiwan Yi-Chen Tsai EIC / FETP Taiwan CDC October 25, 2014 Real-time Outbreak and Disease Surveillance (RODS) system An automated syndromic surveillance system Developed by RODS Lab at University of Pittsburgh since 1999 To collect real-time clinical data from emergency departments (EDs) within a geographic region Applied widely on outbreak detection and decision making in biosurveillance in USA https://www.rods.pitt.edu/site/content/view/13/78/ 2
First introduced RODS in Taiwan (pilot study) Evolution of RODS system in Taiwan System revision Regularly monitor 7 syndromes ~150 hospital EDs Integrated into Infectious Disease Data Warehouse since 2007 Regularly monitor 4 syndromes: EV infection, ILI, Acute diarrhea, Acute hemorrhagic conjunctivitis Replaced Sentinel Physicians Surveillance since 2009 Convert ICD-9-CM to ICD-10-CM in 2015 2004 2005 2006 2007 2008 2009 2010 2015 year Gastrointestinal, Constitutional, Respiratory, Rash, Hemorrhagic, Botulinic, Neurological http://dwweb.cdc.gov.tw/dwweb/ http://nidss.cdc.gov.tw/ 3 Objective To evaluate RODS system in Taiwan Describe system operation Evaluate system performance 4
Evaluation design Data source: RODS data from SAS server and Infectious Disease Data Warehouse ( ) Period and place: 2009/wk1 2013/wk52, nationwide Methods: Interviews: stakeholders and information technologists Review documents Aggregate data of ED visits from National Health Insurance(NHI) database, 2009-2013 Updated Guidelines for Evaluating Public Health Surveillance Systems (MMWR 2001) Framework for Evaluating Public Health Surveillance Systems for Early Detection of Outbreaks (MMWR 2004) 5 Stakeholders Emergency department(ed) and information technology services at hospitals Taiwan CDC Information technology corporation Local health departments Healthcare providers Media The public 6
Purpose of RODS system in Taiwan To monitor trends of syndromic illness for situation awareness in time Enterovirus infection Influenza-like illness (ILI) Acute diarrhea Acute hemorrhagic conjunctivitis (AHC) To provide information for emergent public health events: Animal bites, Hyperthermia 7 Hospitals ED Physician Diagnosis Entry (ICD-9-CM) & Triage information Hospital Information System(HIS) Hospitals set their own data upload schedule (real time - daily) Reporting Server Gateway for data preparation Operation Taiwan CDC User downloads analytics trends of diseases Notifiable Infectious Diseases Statistics System Update analytic data automatically (Once a week, at every Tuesday) Upload data every 5 minutes Data format XML, CSV, DB, HL7 Certificate Validation (ex: HCA) Infectious Disease Data Warehouse Data cleaning, classification, and analysis automatically (Once a day, at 4~8 a.m.) Firewall Receiving Server/ Data Exchange platform 2minutes ED Data Exchange Server Check data format SAS Server Data cleaning, classification, and analysis by SAS software (Once a day, at 2~3 a.m.) 8
Information feedback/ Further actions Provide trends of diseases for policy making Weekly report of epidemic status In Taiwan CDC s meeting To media, the public, healthcare workers Press releases Disease Surveillance Express Influenza Express Weekly Report of Enterovirus Infection 9 Report data format from hospitals Column Name Note Data Required 1 Hospital ID Hospital ID Y 2 PNAME Patient name N 3 PID Patient ID number or passport number N 4 BIRTHDAY Date of birth (yyyy/mm/dd) Y 5 SEX M:male, F:female Y 6 ADMIT_TIME Time of ED visit (yyyy/mm/dd hh:mm:ss) Y 7 SUBJECT Chief complaint N 8 ICD9_1 Y 9 ICD9_2 ICD-9-CM codes N 10 ICD9_3 at least one ICD-9 code is required N 11 ICD9_4 N 12 TRIAGE Triage N 13 TEMPERATURE Body temperature ( ) N 14 STATUS I= new report data, U= modified data Y 10
Funding: Taiwan CDC Resources Personnel at Taiwan CDC 2 full-time IT specialists for system management 5 staffs of EIC for data analysis and interpretation Costs Initial establishment: NTD$ 16 million (Data Exchange Platform establishment) Yearly maintenance: NTD$ 0.46 million No extra charge for enrolling additional hospitals 11 Legal framework of RODS in Taiwan Article 26, Communicable Disease Control Act ( 26 ) Regulations on Implementation of Communicable Disease Surveillance and Alert Systems ( ) 12
Representativeness Year Hospitals participating Level of Hospitals in RODS 2011 178 18 Medical Centers, 76 Regional Hospitals, 84 Local Hospitals 2012 179 21 Medical Centers, 76 Regional Hospitals, 82 Local Hospitals 2013 175 21 Medical Centers, 76 Regional Hospitals, 78 Local Hospitals 13 Number of ED visits from RODS and NHI database, by week of visit, 2009-2013 Year ED visits of RODS (a) ED visits of NHI (b) Ratio (a/b) 2009 5,220,560 6,751,811 77.3% 2010 5,319,287 6,614,022 80.4% 2011 5,675,426 6,690,231 84.8% 2012 6,266,048 6,457,881 97.0% 2013 6,015,030 6,172,176 97.5% Total 28,496,351 32,686,121 87.2% 14
Geographic distribution of hospitals participating in RODS, 2013 In 2013, 168 out of 194 hospital EDs certified for emergency care participated in RODS Coverage rate: 87% Included all medical centers Hospitals participating in RODS, by number of visits per month 15 Acceptability Selectable upload data format for hospitals XML, CSV, DB, HL7 No burden on facilities and manpower after initial setup 16
Simplicity and Flexibility Simplicity: good Automatically collected and analyzed User-friendly interface in Infectious Disease Data Warehouse Flexibility: good Include or exclude hospitals Add or redefine syndromes Able to reuse historical data Share and incorporate data into other systems 17 Timeliness and Stability Timeliness: Excellent Data collection from hospitals: within 10 minutes Automatic data analysis: daily Stability: Good System location & backup site : Taiwan CDC Unscheduled system shutdown: rare Recovery: quick 18
Validity : Data Quality Check required data format automatically Unable to check data contents Avoid duplicated data upload from hospitals Set Hospital ID + Date of Birth + Time of ED visit as the key to identify Completeness of surveillance data Required data for specific variables and check automatically 19 Column Name Data Required Missing data in variables Missing percentage in each period 2009/12 2010/12 2011/12 2012/12 2013/12 Hospital ID Y 0 0 0 0 0 PNAME N 14% 13% 15% 22% 23% PID N 14% 13% 16% 22% 23% BIRTHDAY Y 0 0 0 0 0 SEX Y 0 0 0 0 0 ADMIT_TIME Y 0 0 0 0 0 SUBJECT N 35% 32% 34% 30% 27% ICD9_1 Y 0 0 0 0 0 ICD9_2-52% 50% 49% 51% 50% ICD9_3-79% 77% 76% 77% 76% ICD9_4-93% 92% 92% 92% 91% TRIAGE N 4% 3% 4% 4% 4% TEMPERATURE N 0 0 0 0 0 20
Usefulness - Examples Trends of acute diarrhea syndromic illness and situation awareness in 2012 New public health events of animal bites for re-emergence of rabies in 2013 21 Acute diarrhea surveillance New variant strain of Norovirus (GII,4 2012) identified, which first isolated in March and became the major circulation strain since August, 2012 5 press releases Disease Surveillance Express Health education to the public An unusual rising trend since wk19, 2012 Trend rose obviously since wk40 (Moon Festival), 2012 School open 22
Guidelines for Norovirus Infection Control Health education leaflets 23 http://www.cdc.gov.tw/professional/gastroenteritis Trend of ED visits for animal bites in RODS and PEP applications, 2013-2014 24
Strengths Timely Acceptable Simple Flexible Useful Stability of system facilities Completeness of variables required 25 Weaknesses Internet interruptions: threat for system operation Sensitivity Size of outbreak Data interpretation: challenging Alerts rely on analytical algorithms and personal experience Yet to be evaluated 26
Limitations Lack of good alternate data source to evaluate: Validity of data contents Completeness and stability of data reporting from each hospital Sensitivity & Predictive Value Positive(PVP) 27 Recommendations Evaluate the stability of ED data upload from hospitals Set up automatic evaluation process for the quantity of ED data upload from hospitals Data from hospital EDs should be tested for ICD-9-CM to ICD-10-CM conversion ICD-9-CM will be replaced by ICD-10-CM in 2015 Evaluate alert threshold for situation awareness and data interpretation 28
EIC, Taiwan CDC Chih-Hsi Chang Dr. Yu-lun Liu Shiang-Lin Yang Ni-Chun Yeh Acknowledgement MO/FETP, Taiwan CDC Dr. Wan-Chin Chen Dr. Wan-Ting Huang Dr. Yi-Chun Lo Donald Dah-Shyong Jiang IT, Taiwan CDC Chi-Sheng Hsu Kalen Chung Nancy Chang Cathy Ting 29 THANK YOU FOR YOUR ATTENTION! 30