RESPIRATORY EFFECTIVENESS GROUP RESEARCH PROPOSAL

Transcription

1 RESPIRATORY EFFECTIVENESS GROUP RESEARCH PROPOSAL Development and Database Validation of a Small Item set COPD Management Tool (EXACT-S) for Potential use in Telehealth RESEARCH SUMMARY Through analysis of existing datasets utilising the EXerbAtions of Chronic Pulmonary Disease Tool (EXACT), to develop a reduced (approximately five-item) questionnaire that is sufficiently sensitive and specific to be a practical clinical tool for identifying the onset of exacerbations of chronic obstructive pulmonary disease (COPD) BACKGROUND Exacerbation identification in COPD COPD is a chronic, progressive condition characterised by day-to-day symptomatic variation and exacerbations (episodes of worsening symptoms). Exacerbations are not only associated with reduced quality of life, but also increased healthcare utilisation and a progressive, permanent deterioration in lung function. 1-3 Yet COPD exacerbations are often under-reported 4,5 in part because patients (and clinicians) find it difficult to differentiate exacerbation onset from bad days. Limitations in the accurate identification and differentiation of exacerbations can result in delayed treatment or, conversely, over-treatment. Better differentiation and exacerbation identification is required to inform optimum use of exacerbation therapy such that it minimises the impact on patients quality of life and potential risk of lung function deterioration, while avoiding overuse of antibiotics and steroids. 6 The role of telehealth in COPD The advent of telehealth was initially heralded as a potential solution to improved COPD self-management and exacerbation identification, however, inconsistent results across a range of studies has brought the utility of telehealth management supports under question. 7 Despite the optimism of some early study reports, 8 and a cautiously positive conclusion in a Cochrane review 9 a recent randomised controlled trial of telemonitoring for COPD showed no impact on time to admission, quality of life, or self-efficacy despite a substantial increase in workload due to frequent alerts. 10 Whilst the reasons for this negative outcome are probably multi-factorial, one crucial problem would appear to be the algorithm used. Analysis of the telemonitoring data from a small pilot study revealed that a discrete increase in symptoms was associated with treatment in only about a quarter of patients. 11 The other patients recorded constantly worse symptoms many of whom took frequent extended courses of antibiotics and steroids. Similarly, limitations of current algorithms may explain the variable results seen in trials of self-management for COPD A recent RCT of self-management for COPD observed that less than half the patients seemed able reliably to detect and respond to exacerbations of COPD. 14 1

2 Algorithms for detecting exacerbations have typically been based on the symptoms of breathless, and sputum quantity/purulence used in the classic definitions of exacerbations and still used in international guidelines. 15 Paper-based symptom diaries have been used successfully in a research context, by patients selected as being able and willing to complete daily symptom scores. 1 Telemonitoring data, and evidence from self-management trials suggest that these questions do not reflect symptoms of exacerbations in all patients. Qualitative studies have revealed that patients use a much broader range of symptoms to determine the presence of an exacerbation, 16 which may need to be reflected in questionnaires. An additional challenge is the lack of precision with which exacerbation duration and severity are evaluated (e.g. duration of hospital admission, nature and duration of pharmacological). Event-based definitions cannot account for regional differences in health policy or unreported events, do not standardise the magnitude of change in the patients underlying condition and can be subject to inaccuracies in patients retrospective recall. The EXACT The EXACT was developed under the EXACT-Patient Reported Outcome (PRO) Initiative a multi-sponsor project involving multi-disciplinary experts in response to the recognised need to develop of a single, standardised instrument for evaluating the effects of treatment on acute COPD exacerbations, where exacerbations are considering to be events characterised by an acute, sustained worsening in patients' COPD beyond normal day-to-day variability, including an increase in respiratory symptoms such as dyspnea, cough, and sputum production that may require a change in treatment). 17 Data are captured across the following fourteen fields: cough today ; mucus when coughing ; difficulty with mucus ; chest discomfort ; chest tight ; breathless today ; how breathless today ; shortness of breath [SOB] with personal care ; SOB with indoor ; SOB with outdoor ; weak/tired ; sleep disturbed ; scared/worried. Although the tool has been validated and has demonstrated reliability, sensitivity and acceptability among patients in the clinical trial setting, it is proposed that a shorter version of EXACT would improve its potential utility within routine selfmanagement and telehealth solutions and its ease of use for patients. The first step requires identification of approximately 5 EXACT items that, together, offer optimal sensitivity with adequate specificity to COPD exacerbations and then validating that tool (EXACT-S) within a retrospective database. AIM To use an existing EXACT database to develop a reduced (approximately five-item) questionnaire that is sufficiently sensitive and specific to be a practical clinical tool for identifying the onset of exacerbations of chronic obstructive pulmonary disease (COPD), and to validate the instrument in a second database. EXACT-5 DEVELOPMENT Components There are two key phase to development of the EXACT-S: (i) Item reduction: reduction of the 14-question EXACT to approximately 5 core questions, identified through Rasch statistical analysis (ii) Database validation of the EXACT-S: Using a separate EXACT dataset evaluation of the sensitivity and responsiveness of the EXACT-S to COPD exacerbations (and compared with the 14-question EXACT). 2

3 Data source Item reduction It is proposed that Evidera s EXACT development database (used to develop the original EXACT) be used to carry out the item reduction element of the analysis. Evidera have agreed (via a letter of intent) to use of their anonymised EXACT data for this purpose and to conduct the item reduction analysis phase, in collaboration with an independent Steering Committee made up of REG collaborators (see page 4 for details). EXACT-S database validation Once the EXACT-S has been developed and approved by the Steering Committee, the tool must be validated in a second dataset, held by AstraZeneca. Evidera will develop an analysis plan for how to evaluate the two EXACT-S draft item sets, with input from the EXACT-S Steering Committee. The analysis plan will focus on maximizing sensitivity and adequate specificity in relation to health care utilization events. AstraZeneca will be invited to provide input into relevant variables that are available in their EXACT dataset. The Validation analysis will be conducted in one of two ways: Option 1: AstraZeneca will be responsible for carrying out the proposed validation through analysis of their own EXACT database. Option 2: AstraZeneca will release their EXACT dataset to Evidera and Evidera will conduct the validation analyses. A letter of intent from Evidera indicates willingness on their part to proceed with either option. In Option 1, Evidera s EXACT staff would provide insight and answer questions to support the AZ-lead analysis. In Option 2, Evidera will assign a data analyst to conduct the validation. Methods Item reduction Method: The existing 14 items in the EXACT will be further reduced in number using Rasch analysis. This is an iterative technique that permits removal of one item at a time and then testing for the quality of fit of the remaining items to the overall model. This will reduce the quality of the measurement properties of the instruments, but that is less important than losing sensitivity, however the two could go together. For that reason a second technique will be used to identify the size of change in each item score for events identified by the 14-item set. The investigators will determine the final item set that will comprise the EXACT-S. This requires two value judgments balancing one property against another. For that reason two draft items sets EXACT-S[1] and EXACT-S[2] will be created for further testing on a second dataset. Proposed analyst: Evidera as EXACT development experts and holders of the EXACT development database. Steering Committee Involvement: The EXACT-S Steering Committee will collectively review the data to check the proposed items make sense from a clinical perspective before the validation of the proposed tool commences. 3

4 4

5 EXACT-5 database validation Method: The performance of EXACT-S[1] and [2] will be tested for sensitivity and specificity against exacerbations events identified using the full 14-item set, using data from clinical trials performed by AstraZeneca. A preference will be made for the set with the greatest sensitivity, provided that it has acceptable sensitivity. Proposed analyst: AstraZeneca as owners of a large EXACT database, or Evidera having been granted access to AstraZeneca s EXACT database. Steering Committee Involvement: The EXACT-S Steering Committee will collectively review the validation data with a view to interpreting its clinical significance and potential power prior to publication of the results. OUTPUTS & NEXT STEPS It is proposed that the development of the reduced tool be published as an abstract at key respiratory conferences. On completion of the validation phase of the EXACT-S development, the full analysis, will be published in a peer review journal. If a tool with potential clinical utility is successfully developed, next steps may include developing a telemonitoring-version of the instrument and carrying out a prospective, pragmatic study in which the tool is tested in a pilot population of real-world patients with COPD. If such a pilot study establishes the validity of the EXACT-S, there may be potential to use the instrument in a randomised controlled trial of telemetrically supports selfmanagement. N.B. This proposal relates to the development of the EXACT-S only. Any prospective trial activities are beyond the scope of this document and would commence only if the hypothesis of this study is successfully proven. COLLABORATORS & AGREEMENTS EXACT-S Steering Committee Paul Jones: Professor of Respiratory Medicine, Consultant chest physician, St George s Healthcare NHS Trust Dermot Ryan: Honorary clinical Research Fellow, Centre for Population Health Sciences, University of Edinburgh Hilary Pinnock: Reader, Allergy & Respiratory Research Group, Centre for Population Health Sciences, University of Edinburgh Nancy Leidy: Director of Evidera (to be invited) The collaborators named in this proposal are experts in the fields of: questionnaire development; general practice; respiratory medicine; database management; realworld evaluations; validation studies; development and evaluation of complex interventions; trial management and implementation. Ownership and intellectual property Evidera will retain ownership of the EXACT; to facilitate standardized use and availability, Evidera will be owner of the EXACT-S, with the Respiratory Effectiveness Group and AstraZeneca licensed to use the instrument immediately upon completion of the tool. Evidera is committed to having the EXACT-S easily available for use by all (e.g. instrument, scoring guide, and instructions for use available via website). 5

6 Collaborating organisations Agreements will need to be drawn up between each of the collaborating organizations REG, AstraZeneca and Evidera to reflect their various roles, responsibilities and ownerships, as outlined below: Organisation Nature of involvement Primary contact The Respiratory Effectiveness Group Evidera AstraZeneca Roles & Responsibilities Study Sponsor Proposal development Coordination of & liaison with an Expert Study Steering Group (the EXACT-S Steering Committee) Project management support Data dissemination Ownership: EXACT brand EXACT development dataset Roles & Responsibilities Approval for the Steering Group to develop a reduced version of EXACT Participate as a member of the EXACT-S Steering Committee Permissions to use the EXACT development database to develop the EXACT-S Statistical analysis support to item reduce EXACT to develop EXACT-5 Ownership: EXACT dataset (for EXACT-S validation) Roles & Responsibilities: Provision of EXACT data for database validation of the EXACT-S Statistical analysis support to validate the EXACT- S in the AstraZeneca EXACT dataset Alison Chisholm Nancy Leidy Setareh Williams 6

7 TIMELINES & MILESTONES The following milestone summary indicates the anticipated key elements of the project and anticipated completion dates. The suggested start date for the project is subject to all contracts and permissions being finalised, and the EXACT development dataset being available for analysis, by 1 May Each subsequent analysis phase is anticipated to take approximately 6 months: Milestone Completion Date Contract development & Signing April 2015 Securing approvals & access to required datasets April 2015 EXACT item reduction analysis May October 2015 Expert panel meeting to review EXACT-S (reduced tool) components Abstract Development (focus = item reduction & the reduced tool) EXACT-S sensitivity analysis (evaluation of sensitivity & responsiveness of reduced tool vs EXACT) late September 2015 (ERS Congress) Autumn 2015 (appropriate conference TBC, potentially 2016 ATS/IPCRG/ERS) October 2015 February 2016 Manuscript Development February March 2016 Journal Submission April 2016 BUDGET Milestone Option 1 Option 2 Project Management (including development of contracts & data sharing agreements) Analysis Phase I: item reduction Analysis Phase II: item reduction (including pass through costs) $123,577 $99,417 Expert panel meetings (1 x face-to-face) Congress abstract (development submission; development of presentation materials; presentation) Manuscript Total 7

8 REFERENCES 1. Seemungal TAR, Donaldson GC, Bhowmik A, et al. Time course and recovery of exacerbations in patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med 2000;161: British Thoracic Society. Burden of Lung Disease report: Second edition. Available at http// (last accessed May 2014) 3. Anzueto A. Impact of exacerbations on COPD. Eur Respir Rev 2010;19: Ure J, Hanley J, Pinnock H, et al. Piloting tele-monitoring in Chronic Obstructive Pulmonary Disease: a mixed methods exploration of issues in design and implementation. Prim Care Respir J 2012;21: Adams R, Chavannes N, Jones K, et al. Exacerbations of chronic obstructive pulmonary disease - a patients' perspective. Prim Care Respir J 2006; 15: Herath SC, Poole P. Prophylactic antibiotic therapy for chronic obstructive pulmonary disease (COPD). Cochrane Database of Systematic Reviews 2013, Issue 11. Art. No.: CD DOI: / CD pub2 7. Steventon A, Bardsley M, Billings J, Dixon J, Doll H, Hirani S, Cartwright M, Rixon L, Knapp M, Henderson C, Rogers A, Fitzpatrick R, Hendy J, Newman S; Whole System Demonstrator Evaluation Team. Effect of telehealth on use of secondary care and mortality: findings from the Whole System Demonstrator cluster randomised trial. BMJ. 2012;344:e Bolton CE, Waters CS, Peirce S, et al. Insufficient evidence of benefit: a systematic review of home telemonitoring for COPD. J Eval Clin Pract Dec;17(6): McLean S, Chandler D, Nurmatov U, et al. Telehealthcare for asthma: a Cochrane review. CMAJ Aug 9;183(11):E Pinnock H, Hanley J, McCloughan L, et al. Effectiveness of telemonitoring integrated into existing clinical services on hospital admission for exacerbation of chronic obstructive pulmonary disease: researcher blind, multicentre, randomised controlled trial. BMJ 2013; 347:f Burton C, Pinnock H, McKinstry B. Changes in physiological measures with symptoms and decision to treat exacerbations of chronic obstructive pulmonary disease: analysis of telemonitoring data. J Telemed Telecare 2014 (in press) 12. Zwerink M, Brusse-Keizer M, van der Valk PDLPM, et al. Self management for patients with chronic obstructive pulmonary disease. Cochrane Database of Systematic Reviews 13. Bourbeau J, Julien M, Maltais F, et al. Reduction of hospital utilization in patients with chronic obstructive pulmonary disease: a disease-specific self-management intervention. Arch Intern Med 2003;163: Bucknall CE, Miller G, Lloyd SM, et al. Glasgow supported self-management trial (GSuST) for patients with moderate to severe COPD: randomised controlled trial. BMJ 2012;344:e Global Initiative for Obstructive Lung Disease. Global strategy for the diagnosis, management and prevention of chronic obstructive pulmonary disease See: (last accessed: 10 August 2014). 16. Williams V, Hardinge M, Ryan S, Farmer A. Patients experience of identifying and managing exacerbations in COPD: a qualitative study. npj Primary Care Respiratory Medicine 2014; 24, Article number: Evidera. EXACT-PRO Initiative Website. See: (last accessed: 12 December 2014) 18. Leidy NK, Wilcox TK, Jones PW, for the EXACT-PRO Study Group. Standardizing measurement of chronic obstructive pulmonary disease exacerbations. Reliability and validity of a patient-reported diary. Am J Respir Crit Care Med 2011;183: Petrillo J, Cairns J. Development of the EXACT-U: a preference-based measure to report COPD exacerbation utilities. Value Health. Jun 2011;14: Halpin DM, Laing-Morton T, Spedding S, et al. A randomised controlled trial of the effect of automated interactive calling combined with a health risk forecast on frequency and severity of exacerbations of COPD assessed clinically and using EXACT PRO. Prim Care Respir J 2011;20: