Risk stra)fica)on: The UK cardiothoracic experience

Transcription

1 Risk stra)fica)on: The UK cardiothoracic experience Graeme L Hickey 1 ; Stuart W Grant 2 ; Iain Buchan 1 ; Ben Bridgewater 1,2 1 Northwest Ins.tute of BioHealth Informa.cs, Manchester University 2 Department of Cardiothoracic Surgery, University Hospital of South Manchester

2 Background Around 35,000 adult cardiac surgery procedures performed each year in UK In- hospital mortality rate in was 3.4%

3 What s risk stra)fica)on used for? Governance Decision- making Cardiac Surgery The Society for Cardiothoracic Surgery in Great Britain & Ireland Sixth National Adult Cardiac Surgical Database Report 2008 Demonstrating quality Prepared by Ben Bridgewater PhD FRCS Bruce Keogh KBE DSc MD FRCS FRCP on behalf of the Society for Cardiothoracic Surgery in Great Britain & Ireland Robin Kinsman BSc PhD Peter Walton MA MB BChir MBA Dendrite Clinical Systems

4 Mo)va)on Total cost = 1.48m/year in England (<1% of the total NHS spend on adult cardiac surgery)* Associated with a 50% reduc)on in risk adjusted mortality* *Maintaining Pa.ents Trust, SCTS, Henley- on- Thames: Dendrite Clinical Systems Ltd, 2011

5 Infrastructure Aim: 3 months Reality: 1 year Cardiac surgery Input data locally Aim: <1 year Reality: 3 years Uploaded periodically to central database Cardiac Surgery The Society for Cardiothoracic Surgery in Great Britain & Ireland Sixth National Adult Cardiac Surgical Database Report 2008 Demonstrating quality Prepared by Ben Bridgewater PhD FRCS Bruce Keogh KBE DSc MD FRCS FRCP on behalf of the Society for Cardiothoracic Surgery in Great Britain & Ireland Robin Kinsman BSc PhD Peter Walton MA MB BChir MBA Dendrite Clinical Systems CQC website Na)onal audit Sta)s)cian + clinicians

6 Monitoring methodology 1. Funnel plot Fixed )me period (e.g. 3 years) Iden)fy outlier units Doesn t address whether hospitals are gegng worse 2. Variable life adjusted display (VLAD) plot Intui)ve dynamic summary Doesn t iden)fy when a unit is an outlier

7 0.02 Funnel plot 0.01 Risk- adjusted mortality propor)on Mortality rate 0.00 All elec)ve & urgent meuroscore cardiac surgery (04/07) meuroscore (08/10) in England & Wales meuroscore ( warrants closer inves)ga)on 0.05 Na)onal average ±2σ ±3σ Number of cardiac procedures Number of cardiac procedures Number of cardiac procedures

8 VLAD plot Variable Life-Adjusted Display plot for an individual surgeon Observed Predicted Predicted deaths - observed deaths The bad run The intervention Operation sequence

9 Problems to overcome 1. Systema)c model miscalibra)on 2. Data dissemina)on 3. Pooled vs. separate models 4. Data quality 5. Gaming 6. Subgroup performance 7. Ancillary methodology

10 Systema)c miscalibra)on What s wrong with this? 0.02 Risk- adjusted mortality propor)on Mortality rate rate All elective & urgent cardiac surgery in the UK /07) All elec)ve & urgent meuroscore cardiac surgery (04/07) Logistic EuroSCORE (04/07) in England & Wales meuroscore 3000 (08/10) Number of cardiac procedures Number Number of of cardiac cardiac procedures procedures

11 Systema)c miscalibra)on Observed mortality is decreasing beqer surgical tools improvements in post- surgery treatment Predicted mortality is increasing increase in older pa)ents more complex procedures Model valida)on essen)al! Mortality propor)on Observed Expected Time Actual Overall average Trend

12 Dynamical modeling vs. periodic recalibra)on vs. doing nothing Patient related factors Extracardiac arteriopathy Neurological dysfunction Previous cardiac surgery Age (adjusted) 1.0 Female Pulmonary disease Coefficient Model coefficients (log- odds) Cardiac 0.1 and operation related factors 0.0 Serum creatinine Unstable angina > 200µmol/l 1.2 LV Active function: endocarditis moderate LV Critical function: pre op poor Time Model: Model 1 Model 2 Model 3

13 Data dissemina)on: past Abandoned CQC website The SCTS Blue Book 512 pages!

14 Data dissemina)on: future EuroSCORE series Cummulative meuroscore Mr Ben Bridgewater meuroscore Date Total number of deaths Cumulative mortality Predicted Observed VLAD (with date dispersion) Cummulative Mortality Unit of Interest Date Date VLAD Crude mortality funnel plot Risk adjusted mortality funnel plot Mortality rate 0.05 Mortality rate Number of cardiac procedures Number of cardiac procedures Date

15 Data dissemina)on: future

16 Data quality Outlier surgeon rogue surgeon Number of incomplete records Number of procedures Missing data Input souware errors Registry cleaning errors Imputa)on Valida)on

17 Pooled vs. separate models CABG + MVR + Tricuspid repair = AVR? Cardiac surgery is a catch- all term We could have risk predic)on models for: 1. all procedures (combina)ons) 2. all procedures with mul)ple procedure variables 3. each procedure group (e.g. CABG, Valve, CABG + Valve, ) Decision depends on applica)on.

18 Gaming (+ other unexpected extraneous varia)on) Rank ?? Hospitals Hospital BAL. Barts and The London BAS. Basildon Hospital BHL. Liverpool Heart and Chest Hospital BRI. Bristol Royal Infirmary CHH. Castle Hill Hospital CHN. Nottingham City Hospital ERI. Royal Infirmary of Edinburgh FRE. Freeman Hospital GEO. St George's Hospital GJH. Golden Jubilee Hospital GRL. Glenfield Hospital HAM. Hammersmith Hospital HH. Harefield Hospital HHW. Wellington Hospital North HSC. Harley Street Clinic KCH. King's College Hospital LBH. London Bridge Hospital LGI. Leeds General Infirmary MOR. Morriston Hospital MRI. Manchester Royal Infirmary NCR. New Cross Hospital NGS. Northern General Hospital NHB. Royal Brompton Hospital PAP. Papworth Hospital PLY. Derriford Hospital QEB. Queen Elizabeth Hospital RAD. John Radcliffe Hospital RIA. Aberdeen Royal Infirmary RSC. Royal Sussex County Hospital RVB. Royal Victoria Hospital SCM. James Cook University Hospital SGH. Southampton General Hospital STH. St Thomas Hospital STM. St Marys Hospital Paddington STO. University Hospital of North Staffordshire UCL. University College Hospital UHW. University Hospital of Wales VIC. Victoria Hospital WAL. University Hospital Coventry WYT. Wythenshawe Hospital Distribu)on of ranks of risk factor prevalence might be expected to homogenous across hospitals Further inves)ga)on required

19 Subgroup performance Stra)fica)on does not ensure good model performance

20 Ancillary methodology Mul)ple tes)ng correc)on adjustments (e.g. Bonferroni) Overdispersion mul)plica)ve variance infla)on random effects models