Project Brief New Cancer Waiting Times Data Quality Assurance Audit Version 1.0
Contents 1 Introduction...3 2 Data Recording and Submitting...4 3 Data Quality Assurance Audit...4 3.1 Areas of Investigation:...4 3.2 How the DQA audit will be carried out:...5 3.3 Sampling:...6 4 Analysis & Reporting...6 Appendix 1 - Dataset...7 Appendix 2 NHS Board Tracking Systems...9 Appendix 3 - Dataflow...10 Appendix 4 - Timeline of Audit...11 Appendix 5 NHS Board Sample Size...12 Appendix 6 - NHS Board Exclusions...13 Version 1.0 2
1 Introduction In October 2008, the Scottish Government published Better Cancer Care - An Action Plan, where it announced it would: Extend the 62-day urgent referral to treatment target to include patients who had screened positive and all patients urgently referred with a suspicion of cancer (to be delivered by 2011). Introduce a new 31-day target for all patients diagnosed with cancer (no matter how they were referred) from decision to treat to treatment (to be delivered by 2011). The statements in the action plan form the basis for the revised set of cancer waiting targets, which are as follows: 62-day target from receipt of referral to treatment for all cancers. This applies to each of the following groups: o o o any patients urgently referred with a suspicion of cancer by their primary care clinician (for example GP) or dentist any screened-positive patients who are referred through a national cancer screening programme (breast, colorectal or cervical) any direct referral to hospital (for example self-referral to A&E) 31-day target from decision to treat until first treatment for all cancers, no matter how patients are referred. For breast cancer, this replaces the existing 31-day diagnosis to treatment target. Initially these targets will cover the following cancer types: Breast Colorectal Head & Neck Lung Lymphoma Ovarian Melanoma Upper Gastro-Intestinal (hepato-pancreato-biliary (HPB) and oesophago-gastric (OG)) Urological (prostate, bladder, other) Cervical Further cancer types may be added in future. The existing 31-day acute leukaemia and paediatrics standards (formerly targets) and their reporting continue unchanged. A tolerance level is applied to the new targets. The agreed tolerance level is 5% for each of these targets, that is the stated waiting time must be met for Version 1.0 3
95% of all patients covered by the 31- and 62-day targets. For some patients it will not be clinically appropriate for treatment to begin within the target timescales. 2 Data Recording and Submitting Approx 1800 records from all NHS Boards are submitted each month. The d ataset contains 21 fields mostly mandatory (see Appendix 1). Data is collected by cancer trackers and their role, in terms of New Cancer Waiting Times (NCWT), is to find all urgent GP referrals with a suspicion of cancer, A+E episodes and screen detected cancer patients detected through the national breast, bowel and cervical screening programmes. In most NHS Boards, the trackers have a role in managing the patient journey through the system, expediting all local outpatient investigations and appointments and treatments. The trackers record relevant patient information obtained from Patient Administration Systems (PASs) and Laboratory systems and record it on the NHS Boards tracking system (systems vary between NHS Boards please see Appendix 2). From this the Information Services Division (ISD) template for submission of NCWT data is completed and is based on the date of treatment. The information is then submitted to ISD via SWIFT DVL (see Appendix 3) for the previous calendar month. Once the submission is received by ISD, the data is validated and uploaded on to Business Objects to allow data interrogation and reporting. 3 Data Quality Assurance Audit The Data Quality Assurance (DQA) team in ISD will be conducting a quality assurance audit of the new cancer waiting times. The audit is required to ensure that data being recorded is accurate and that recording is consistent across Scotland. The audit will support the NHS Boards in determining any areas where data quality can be improved and help them resolve any issues. The audit will contribute towards the fitness for publication exercise and help ensure that when recording and submitting data for publishing national statistics that data quality is taken into account. Appendix 4 shows the expected timeline of the Audit. Future data quality audits will be carried out in the future. 3.1 Areas of Investigation: The DQA team will investigate the following areas and determine If correct codes are being used. If waiting times adjustments and exclusions (patient refused treatment, patient died before treatment or clinically complex) are being applied correctly and consistently, and whether there is any supporting evidence on the NHS Boards systems. Version 1.0 4
If the rules and guidance are being appropriately applied. 3.2 How the DQA audit will be carried out: Data will be audited at an NHS Board level and will be applicable to mainland NHS Boards only (Island NHS Boards will be excluded due to the low number of records submitted by the NHS Boards and the resource of carrying out the audit in the Islands). The audit will look at the 62 and 31-day targets together and will be based on NHS Board of Receipt of Referral for patients treated in Quarter 2 (April-June 2010 data as published on 28 September 2010). Patients who are treated outwith their referring NHS Board will also be included in the audit. Screening and cervical cancer patients will be excluded from the audit as data has not yet been published on these cancer groups. Golden Jubilee National hospital will be included in the audit for patients treated within the 31-day pathway (as the 62-day target does not apply the Golden Jubilee). The following fields will be looked at: Date of Receipt of Referral Urgency and Source of Referral Cancer Type Date Decision to Treat Date of First Treatment Waiting Time Adjustment Pre-DTT - Number of Days Waiting Time Adjustment Pre-DTT - Main Reason Waiting Time Adjustment Post-DTT - Number of Days Waiting Time Adjustment Post-DTT - Main Reason Reason for Delay 62-day Reason for Delay 31-day In addition the following derived items, calculated from data fields submitted by NHS Boards, will be looked at to determine exclusions have been applied correctly: Receipt of Referral Not Recorded Receipt of Referral Inapplicable DTT Not Recorded Patient Refused All Treatment Patient Died Before Treatment Date of Death Date of Treatment Refusal First Treatment Not Recorded Version 1.0 5
Pre-assessment and post-assessment discussions will be held between DQA and NHS Boards. NHS Boards may wish to invite their clinicians to the post-assessment discussions 3.3 Sampling: A sample case listing based on all referrals for Quarter 2 (April June 2010) will be generated for each NHS Board (based on Board of Receipt of Referral). The sample will exclude cervical cancer and screened patients, and also the Island NHS Boards. Some patients treated outwith the Board of referral will be included in the sample. The case listing will not include excluded patients; they will be examined separately, resulting in two case listings being sent to each NHS Board (please see appendix 5 and 6). Sample case listing It has been decided to sample 372 referrals based on a 5% margin of error, 95% confidence; this equates to 8.5% of all referrals (excluding Island NHS Boards, cervical cancer, screened and excluded patients). The size of sample within each NHS Board has been weighted according to the percentage of referrals submitted by each NHS Board in Quarter 2 see Appendix 5. The random sample will include some cases where a Waiting Times Adjustment has been applied. Extra patients have been added, where required, to ensure that the minimum sample size per health board is no less than 20 records. The sample case listing will be generated randomly using SPSS (statistical software package). Exclusions case listing In Quarter 2, there were only 114 exclusions in total; to ensure that an adequate audit of these exclusions is carried out, it has been agreed to look at all exclusions for each NHS Board (excluding Island NHS Boards, cervical cancer and screened patients) see Appendix 6. Both case listings will be produced by the New Cancer Waiting Times (NCWT) team and supplied to the DQA team. The listings will include CHI, Cancer Type, Type of first treatment, NHS Board of Receipt of Referral and NHS Board of First Treatment. It is recognised that some case notes may not be available at each site visit; therefore the NCWT team will include an extra 50% of cases in the sample case listing. This will ensure that the minimum number of cases can be examined. The cases will be prioritised to ensure a proportionate number of records containing waiting time adjustments are available for auditing. 4 Analysis & Reporting Analysis will be carried out jointly by DQA and the NCWT team at ISD. A preliminary report of the detailed findings will be issued to the relevant NHS Board lead contact after the assessment for comments and discussion. Once Version 1.0 6
agreed by the NCWT Team, DQA and the NHS Board lead contact, a final report will be issued to the relevant Chief Executive, with recommendations for action. ISD will also share these findings with the Scottish Government. In addition, a high level summary report of the findings detailing the results for all cancer types combined by NHS Board and Network level will then be published on the NCWT webpages. As detailed in Appendix 4, it is anticipated that the high level summary report will be published in June 2011. As the number of samples cases for GJNH may be relatively small a summary of findings document will be compiled. Appendix 1 - Dataset Field Format Mandatory CHI Number 10 numbers Yes Unique Care Pathway Number TBC No Patient Hospital Patient Identifier 10 characters No Demographics Patient Postcode at Referral 7 characters Yes Ethnic Group 2 characters Yes Date of Receipt of Referral DDMMYYYY Yes, if a patient is in the 62-day target cohort Referral Board of Receipt of Referral 1 character Yes Urgency and Source of 2 characters Yes Referral D iagnosis Cancer Type 2 characters Yes Decision To Date Decision to Treat DDMMYYYY Yes Treat Type of First Treatment 2 characters Yes First Treatment Date of First Treatment DDMMYYYY Yes Board of First Treatment 1 character Yes Waiting Time Adjustment Pre- Number Yes DTT - Number of Days Waiting Time Adjustment Pre- 2 characters Yes Waiting Times Adjustment DTT - Main Reason Waiting Time Adjustment Number Yes Post-DTT - Number of Days Waiting Time Adjustment 2 characters Yes Adjusted Length Post-DTT - Main Reason Referral to Treatment (days) Number Yes, if a patient is in the 62-day target cohort of Pathway Decision to Treat to Treatment (days) Number Yes Version 1.0 7
Reason for Delay 62-day 2 characters Yes Reason for Delay Reason for Delay 31-day 2 characters Yes Version 1.0 8
Appendix 2 NHS Board Tracking Systems Cancer Network NHS Board Systems Currently Used Other systems Used WOSCAN Ayrshire and Arran Excel spreadsheet Glasgow tracking system (links to Topas and SCI Gateway) expected in late 2010 SCAN Borders Not known Dumfries and Tracking system SCAN Galloway links to Topas SCAN Fife e-oasis WOSCAN Forth Valley Access database N/A NOSCAN WOSCAN NOSCAN Golden Jubilee Grampian GG&C Highlands Excel spreadsheet/access database Cancer care pathway database In-house Tracking system In-house Tracking system WOSCAN Lanarkshire Tracking system SCAN Lothian Trak collected from topas, cancer referral box (information obtained from SCI Gateway), SCI Store, EDMS, RIS, E-ward Helix Sci store, SCI Gateway, PAS, Labs, Unisoft, Excelicare PAS, Isoft PMS Patient Management system Carestream Radiology Labarotory Systems All Labs UNISOFT GI Investigations SCI Referrals SCI Store Retrieval of reports from other boards Cancer waiting times database ORMIS Theatre system NOSCAN Orkney In-house Tracking system Orkney patients with pathology reports are entered on NHS Grampian system NOSCAN Shetland Cancer Care Pathway (CCP) ISOFT Express (Grampian PAS) SCI Outpatients SCI Store (both Shetland & Grampian) SCI Gateway APEX EDIS ECASE Helix PMS (Local PAS) NOSCAN Tayside Tracking System TURS Tayside Urgent Referral System. e-case: Cancer Audit data collection system NOSCAN Western Isles Topas system Version 1.0 9
Appendix 3 - Dataflow PAS/PMS system Lab Reports Screening systems Tracking system Resubmission of enitre treatment month's data e.g. extra patient identified or record already submitted requires to be amended. Submission Template File checked for errors SWIFT DVL Oracle Database Business Objects File interrogated for duplicates and inappropriate recording Quarterly Reports generated Monthly reports generated Version 1.0 10
Appendix 4 - Timeline of Audit Month of work (2010/11) Work Phase August September October November December January February March April May June Preparation of audit DQA visits NHS Boards to carry out Audit Analysis Reporting Discuss reports with NHS Boards Publication of reports on NCWT website 11
Appendix 5 NHS Board Sample List Sample 1 Data Quality Assurance Audit - New Cancer Waiting Times Recommended sample size based on all referrals in Quarter 2 (2010) All referrals* Original recommended sample size (5% error) % of all referrals* Final Recommended sample size (5% error)** Scotland 4,390 354 372 NHS Board (Receipt of referral) NOSCAN NHS Grampian 427 34 9.7% 34 NHS Highland 269 22 6.1% 22 NHS Orkney x x x NHS Shetland x x x NHS Tayside 384 31 8.7% 31 NHS Western Isles x x x SCAN NHS Borders** 91 7 2.1% 20 NHS Dumfries & Galloway** 177 14 4.0% 20 NHS Fife 294 24 6.7% 24 NHS Lothian 642 52 14.6% 52 WOSCAN NHS Ayrshire & Arran 319 26 7.3% 26 NHS Forth Valley 285 23 6.5% 23 NHS Greater Glasgow & Clyde 1,055 85 24.0% 85 NHS Lanarkshire 447 36 10.2% 36 Total sample size proportion 8.1% 8.5% * Referrals exclude the following: cervical cancer and screened patients, Island NHS Boards and 'Excluded' cases ** Extra cases have been added to NHS Borders and NHS Dumfries & Galloway to ensure a minimum sample size per NHS Board of 20 records. 12
Appendix 6 - NHS Board Exclusion List Breakdown of exclusions in Quarter 2 (2010) by NHS Board (Receipt of referral) Number of exclusions* % of exclusions Scotland 114 NHS Board (Receipt of referral) NOSCAN NHS Grampian 2 1.8 NHS Highland 6 5.3 NHS Orkney x x NHS Shetland x x NHS Tayside 14 12.3 NHS Western Isles x x SCAN NHS Borders 0 0.0 NHS Dumfries & Galloway 1 0.9 NHS Fife 13 11.4 NHS Lothian 22 19.3 WOSCAN NHS Ayrshire & Arran 7 6.1 NHS Forth Valley 19 16.7 NHS Greater Glasgow & Clyde 22 19.3 NHS Lanarkshire 8 7.0 * The table excludes Island NHS Boards, cervical cancer and screened patients. 13