Validation of a Simple Screening Tool for Identifying Unmet Palliative Care Needs in Patients With Cancer

Transcription

1 Health Care Delivery Original Contribution Validation of a Simple Screening Tool for Identifying Unmet Palliative Care Needs in Patients With Cancer By Paul A. Glare, MD, and Kimberly Chow, ANP-BC Memorial Sloan Kettering Cancer Center, New York, NY Abstract Purpose: The National Comprehensive Cancer Network (NCCN) palliative care (PC) guideline recommends PC screening for all patients and provides criteria for identifying those in need of referral to a PC specialist. This two-step process has not yet been validated as accurately identifying patients in need. The aim of this study was to validate a simplified method for screening and referral based on the existing guideline criteria. Methods: An 11-item screening tool was created, with scores from 0 to 14. Content validity was assessed by a panel of local PC experts. Construct and criterion validities were evaluated using data obtained from a previous study of guideline-based screening and referral. Results: Content validity of the tool was high, with eight items rated as essential. Patients who were closer to death had significantly higher scores, indicating its construct validity. Scores were also higher in patients who were identified as needing a consult and in those who had worse pain and other symptoms, indicating its criterion validity. Using a score of 5 as the trigger, approximately one third of hospitalized patients in the previous study would have been referred to a PC specialist, twice as many as occurred when the attending oncologist relied on his or her clinical judgment. Conclusion: The tool seems to be a valid method for identifying patients with cancer with complex PC needs who would benefit from a PC consult. Reliability testing, external validation, and demonstration of the utility of the tool as a decision aid all await confirmation. Introduction Now that most cancer centers and other hospitals have a palliative care (PC) program, 1 questions arise regarding which patients should be seen in these programs and when. The benefits of early integration of PC with standard oncologic care have been demonstrated. 2 Although it may be theoretically optimal for PC specialists to take on all aspects of PC, some core elements of PC as is the case in any medical discipline should be routine aspects of care delivered by any practitioner. 3 These include aligning treatment with a patient s goals and basic symptom management. Other skills are more complex and take years of training to learn and apply, such as negotiating a difficult family meeting, addressing veiled existential distress, and managing refractory symptoms. Multiple barriers to referral to a specialist PC program exist, including oncologist nonawareness of the program, what the program can do, when to refer, and oncologist unwillingness to refer. 4 To help overcome these barriers, the National Comprehensive Cancer Network (NCCN) recommends that all patients with cancer be screened for PC needs at their initial visit and rescreened at appropriate intervals and as clinically indicated. Consultation or collaboration with PC specialists is recommended for patients with more complex problems. 5 Although the NCCN PC guideline recommends screening, there have been few studies evaluating the logistics of screening oncology patients on a large scale. This may not be surprising, given that the screening advocated in the guideline involves a comprehensive evaluation of the patient and family across six domains, including: one, pain and other symptoms; two, psychological distress; three, comorbid physical and psychosocial conditions; four, treatment options; five, patient or family concerns about disease course and decision making; and six, prognosis. To add to the complexity of screening, the NCCN posits an additional checklist of 24 physical or psychosocial problems for determining which patients have more complex cases and should be considered for consultation or collaboration with a specialist PC program. To evaluate the feasibility of implementing the screening and referral components of the guideline, testing took place in both hospitalized and ambulatory patients in one medical oncology service at Memorial Sloan Kettering Cancer Center (MSKCC; New York, NY). 6,7 To facilitate the complex screening process advocated by the guideline, two pencil-and-paper tools were developed, one for the seven initial screening items and the other for the supplementary list of 24 referral criteria. Although these projects demonstrated the feasibility of screening and the prevalence of PC problems in both the inpatient and outpatient settings, questions arose regarding the validity and reliability of the screening tools that had been created. In particular, there seemed to be redundancy in some of the items on the two lists. Simplifying screening to a one-step validated process seems to be desirable before further work is done in this area. The purpose of this article is to describe a one-step screening tool derived from the two-step guideline, validated against the data obtained from the previously published study of inpatients at MSKCC. 7 Methods Patient Population A total of 194 patients who were hospitalized in the MSKCC GI Oncology Service during the winter of 2010 to 2011 were screened for PC needs. 7 The population of screened patients Copyright 2014 by American Society of Clinical Oncology JANUARY 2015 jop.ascopubs.org e81

2 Glare and Chow Table 1. Screening Tool, Scoring System, and CVRs for Each Tool Item Criterion Points* CVR Locally advanced or metastatic cancer Functional status of patient (ie, ECOG score) Any serious complication of cancer associated with survival 12 months Any serious comorbidity Any other condition complicating care Additional criteria Uncontrolled symptoms Moderate-severe distress Patient/family concerns regarding decision making Team needs assistance with decision making Patient/family requests PC consult Prolonged length of stay Total 0-14 Abbreviations: COPD, chronic obstructive pulmonary disease; CVR, content validity ratio; ECOG, Eastern Cooperative Oncology Group; KPS, Karnofsky performance score; NCCN, National Comprehensive Cancer Network; PC, palliative care; PCCB, Palliative Care Center of the Bluegrass (Lexington, KY). * Scoring system recommended by PCCB; cut point for consult recommended by PCCB was score or 3. Examples given in NCCN guideline include: ECOG of 3 or KPS of 50, hypercalcemia, brain or cerebrospinal fluid metastasis, delirium, superior vena cava syndrome, spinal cord compression, cachexia, malignant effusions, bilirubin 2.5 mg/dl, and creatinine 3 mg/dl. NCCN guideline does not specify these conditions. PCCB suggests: liver disease, moderate or end-stage renal disease, moderate or advanced cardiac disease, moderate or advanced COPD, stroke with loss of 50% of function, other life-limiting illnesses, or other conditions complicating care. was younger than usual, with only one third age 65 years. More than 90% had advanced disease, and 60% were still pursuing active treatment. Admissions were mostly for the management of the complications of advanced cancer or treatment-related toxicities; almost none were for administration of chemotherapy. A waiver to use existing patient data was previously obtained from the MSKCC Institutional Review Board to analyze and disseminate the results of the quality improvement project for which they were collected. A separate waiver was not sought to reuse the same data for the purposes of validating the tool used in the project. Determination of Tool Validity The screening tool is summarized in Table 1. It comprises 11 items covering five clinical dimensions, which define the need for PC in patients with cancer: extent of disease, performance status (PS), prognosis, comorbidities (with two subparts), and PC-specific problems. Three versions of validity of the tool were evaluated: content validity, construct validity, and the two forms of criterion validity. 8 Internal consistency of the individual items was also calculated, but other aspects of reliability (test-retest reliability and inter-rater reliability) could not be evaluated from this data. All calculations were performed in Excel (Microsoft; Redmond, WA) using the data analysis add-in function. Content validity evaluates if a scale or other measurement incorporates the various domains of the phenomenon it purports to measure. In this case, the screening tool would have content validity if its items were considered to be relevant for capturing the kind of problems that are within the scope of practice of a specialist PC program. Although this may be done as a subjective exercise, we attempted to quantify content validity by calculating the content validity ratio (CVR), being the extent of agreement among content experts on how essential a particular item in the tool is, using the following categories: essential to be included; useful, but not essential to be included; or not necessary to be included. The CVR score for an item can range from 1to 1; a positive value for an item in the CVR accords it at least some content validity. The mean CVR across items may be used as an indicator of overall content validity. Seven physicians and nurse practitioners from our 13-member interdisciplinary team (IDT), all certified in palliative care, returned completed surveys, which rated the essentiality of the items in the tool and the NCCN guideline screening items and referral criteria. Construct validity evaluates the extent to which a measurement corresponds to theoretic concepts concerning the phenomenon under study. PS falls and symptom burden rises as death approaches in patients with advanced cancer, 9 so the screening tool score ought to be higher in patients with a shorter survival, if the tool has construct validity. Analysis of variance was used to compare the scores on the screening tool for patients who survived 3,3to6,and 6 months. Internal consistency, which assesses the consistency of results across the items of a tool, was evaluated as another marker of construct validity using Cronbach s alpha. Criterion validity evaluates the extent to which the measurement correlates with an external criterion or gold standard for the phenomenon under concern. This is a challenging concept for health care services that are not disease based, such as PC. In this case, symptom scores and consultation requests were used to test for concurrent validity and predictive validity, which together satisfy criterion validity. Concurrent validity was tested by comparing patient self-reports for pain intensity and symptom distress, using the condensed Memorial Symptom Assessment Schedule (CMSAS), 10 with the identification of uncontrolled symptoms and moderate-severe distress by the screening tool. Predictive validity was evaluated by testing if the total score on the tool was higher in patients identified as needing consultation whether based on the clinical judgment of the on-service oncologist or triggered by meeting one NCCN referral criteria. Results Content Validity The items in the screening tool and the referral criteria came directly from a national panel of experts convened by the NCCN and so would be expected to have strong content validity, although no data are available for how this list was developed or if a CVR was calculated for the NCCN panelists. The e82 JOURNAL OF ONCOLOGY PRACTICE VOL. 11, ISSUE 1 Copyright 2014 by American Society of Clinical Oncology

3 Simple Screening Tool for Identifying Unmet Palliative Care Needs Average Score CVR for the individual items in the screening tool is listed in Table 1. Nine of the 11 items in the tool were rated as essential by the survey respondents; two items had negative CVRs (ie, useful but nonessential); no member of the IDT rated any of the items as unnecessary for inclusion. When the NCCN guideline lists were evaluated, each of the screening items was rated as essential by a majority of the IDT members, but only eight of the 24 items on referral criteria were rated essential. There was a large degree of overlap between the eight essential referral criteria and the screening items. Nine items on referral criteria were considered unnecessary by at least one member. Construct Validity The average score on the tool was significantly higher in patients who survived 6 months (5.0 v 3.2; t ; df 228; P.001). Furthermore, as shown in Figure 1, the average screening tool scores were also significantly different across the three survival groups ( 90, 91 to 180, and 180 days), being 5.3, 3.9, and 3.3 points, respectively (SS ; df 2; MS ; P.001). The internal consistency of the tool was high, with the extent of agreement in scores across the tool exceeding 80% ( 12; E variance 1.32; variance 5.083; Cronbach s alpha.808), further supporting that the items in the tool are addressing a unified construct. Criterion Validity 90 days days > 180 days Figure 1. Construct validity: higher tool score associated with shorter survival. Concurrent validity. Complete pain scores and CMSAS scores were available for 122 admissions, including 48 of 100 admissions of patients who were identified as having uncontrolled symptoms on screening and 74 of 129 who did not. Although a lot of data were missing, those screening positive for uncontrolled symptoms had a higher average pain intensity score ( standard deviation [SD]; v ; t 2.08; df 136; P.04) and a higher physical symptom distress score on the CMSAS ( v ; t 3.29; df 120; P.001). However, the average psychological symptom score ( SD) was no higher in patients who screened positive for being distressed than those who were not ( v ; P.7). Predictive validity. The score on the tool was higher in those patients who were identified as in need of a consult by both methods (ie, oncologist judgment and NCCN referral criteria). The score ( SD) was versus (P.01) when oncologists used their clinical acumen and versus (P.001) for the NCCN referral criteria. Using a score of 5 on the tool as the cut point for triggering a consult, a consult would have been recommended for 34% of screened patients. A score of 5 had a high positive predictive value (PPV; 80%) that a patient would meet one of the NCCN referral criteria but only a moderate negative predictive value (NPV; 44%), meaning that almost half the patients meeting the NCCN criteria would be missed. On the basis the oncologists decision to call a consult, a score of 5 had a PPV of only 25%, suggesting that the clinicians were missing many of the patients who needed a consult. However, the NPV in this setting was 89%, indicating that if the score was 5, it was unlikely a consult would be needed. Discussion This evaluation indicates that this one-step, scored screening tool based on the NCCN PC guideline is a valid method for identifying patients with cancer who might benefit from a PC consult. At a time when workforce issues in both oncology and palliative care are becoming more pressing, 3 this is an important tool for busy oncologists to have at their disposal. Validation was performed from multiple perspectives. A majority of a panel of local PC experts concurred that most of the items in the tool were essential to be included. Patients who were closer to death had higher scores on the tool. Patients who had more pain and other symptoms had higher scores, although there was no association between psychological symptoms and screening positive for distress. Scores were higher on the tool in patients for whom a consult was requested, whereas a low score on the tool had a high NPV for consultation. Using a cut point of 5, approximately one third of patients would be referred to a PC specialist twice as many as currently occurs when oncologists rely on their subjective judgment. The implementation study indicated that these consults were called upstream, occurring at an earlier time and when patients were less distressed. 7 Earlier integration of PC with oncology care allows more time for the benefits of PC to be achieved, compared with a PC consult being called in reaction to a pain crisis or a goals-of-care emergency at the time of hospitalization. Other PC screening tools have been developed, 4,11-13 including the tool from which this one was adapted, 4 but few have undergone validation or other kinds of evaluation. An exception is the NEST (Needs of a Social Nature, Existential Concerns, Symptoms, and Therapeutic Interaction) 13 tool. Although it facilitated greater documentation of illness-related needs than routine clinical assessment, it was less clear that using the NEST 13 produced changes in clinician response, such as calling more PC consults. 12 A study presented at the 2013 American Society of Clinical Oncology Quality of Care Symposium 14 described a similar screening tool to that reported here, further supporting the content validity. Widespread implementation of a validated screening tool could have significant implications for the PC workforce, which Copyright 2014 by American Society of Clinical Oncology JANUARY 2015 jop.ascopubs.org e83

4 Glare and Chow is already strained (Appendix, online only). Extrapolating data from Ontario, Canada, where all ambulatory patients attending cancer centers have their PS and symptom burden scored at each visit, 15 to the United States (1.5 million new cancer diagnoses per year), approximately one in eight new visits (187,500 patients) will result in a score 5 (ie, positive screening). Approximately half of these patients would be expected to die within the next 6 months. 9 If these patients are seen in clinic monthly, 2 screening would result in an extra 1.5 million physician-patient encounters per year. Assuming a full-time equivalent (FTE) PC specialist may see 15 patients per day, approximately 400 additional FTE PC specialists would be needed to be employed for this purpose alone, aggravating the existing specialist PC physician FTE shortage of 8,000. This strengthens the case for a division of labor between generalists and PC specialists. 3 Aside from workforce issues, a potential clinical limitation of setting the cut point at 5 is that all patients with metastatic cancer and an Eastern Cooperative Oncology (ECOG) PS of 3 would screen positive. However, it is important to emphasize that this is a screening tool, not an automatic consult trigger, so it does not mean that all patients with an ECOG PS of 3 must have a consult. Rather, it should prompt the oncologist to consider a PC consult if he or she has not already done so and discuss this with the patient and family. In the Japanese study of outpatient screening, only 50% of patients who screened positive proceeded to PC consultation. 13 Nevertheless, as a group, patients with metastatic cancer and an ECOG PS of 3 have a survival typically measured in a small number of months 16 and increasing symptom burden and distress, 9 so it is likely that a PC specialist would be able to assist in the care of many of these patients. 13 If oversensitivity remains a concern, another option would be to borrow from cardiology and divide the items into major (ie, extent of disease, PS, complications, comorbidities) and minor criteria (ie, physical symptoms, distress, goals of care), as is done for rheumatic fever, 17 and develop rules for referral. 17 Beyond PC to the broader issue of tools for the systematic needs assessment of patients with cancer, a review published in 2000 identified nine questionnaires for the assessment of patients needs and six for the assessment of needs of family members. Most of the instruments were carefully constructed, with satisfactory validity and reliability. 18 However, the need for care was often confounded by satisfaction with care and the problems experienced by patients. Only one questionnaire for patients specifically addressed the need for help; none for family members was so specific. Data on the feasibility of questionnaires for use in regular care were scarce. Issues frequently omitted were spiritual issues, the personal needs of family members, and the continuity of care. It also seemed that most instruments had been constructed for research purposes and had not been tested for use in practical care. Seven years later, another review identified 15 tools that were appraised for psychometric properties, such as validity, reliability, responsiveness to change, and feasibility, including acceptability to patients. 19 Each tool met some of these criteria, but none was found to be complete for all of them. Again, data on how feasible they were to use in practice were scarce, and it was concluded that most were insufficiently tested to recommend for use in routine care. Now that our tool has been validated, its other psychometric properties need to be tested. The previous study of its use in hospitalized patients indicated that although it was quick, easy to use, and effective, there were concerns regarding its acceptability by nurses and oncologists and regarding the accuracy and reliability of the scoring, because many of the items in the tool are subjective or vaguely defined. These issues need further evaluation; they could potentially be resolved by developing an electronic version of the tool that self-populates with clinical data from the electronic health record, supplemented with patient and family self-reports of symptoms, distress, and other concerns. This screening tool could also be adapted for use in special areas, such as the intensive care unit, by replacing the items related to cancer and its complications and comorbidities with commonly used physiology scores and prognostic tools paired with a prolonged length of stay without clinical progress, although revalidation would be required. In conclusion, this simple 11-item screening tool has been shown to be a valid method for screening for PC problems in patients with cancer hospitalized at a comprehensive cancer center. Consistent with the concept of screening in health care, its scoring system provides clinicians with a means to identify which of their large volume of patients should be considered for PC referral. Using a score of 5 as the trigger, 34% of patients who were screened would be recommended to receive a consult, with a high PPV that they would meet one of the NCCN referral criteria. Now that it has been shown to be valid and feasible to use in routine caregiving, the impact of screening with the tool on quality of care needs to be established. Acknowledgment We thank our physician and nurse practitioner colleagues who participated anonymously in evaluating the content validity of the tool. Authors Disclosures of Potential Conflicts of Interest The authors indicated no potential conflicts of interest. Author Contributions Conception and design: Paul A. Glare Collection and assembly of data: Paul A. Glare Data analysis and interpretation: All authors Manuscript writing: All authors Final approval of manuscript: All authors Corresponding author: Paul A. Glare, MD, Palliative Medicine Service, Department of Medicine, Memorial Sloan Kettering Cancer Center, 1275 York Ave, New York NY 10065; glarep@mskcc.org. DOI: /JOP ; published online ahead of print at jop.ascopubs.org on November 12, e84 JOURNAL OF ONCOLOGY PRACTICE VOL. 11, ISSUE 1 Copyright 2014 by American Society of Clinical Oncology

5 Simple Screening Tool for Identifying Unmet Palliative Care Needs References 1. Hui D, Elsayem A, De la Cruz M, et al: Availability and integration of palliative care at US cancer centers. JAMA 303: , Temel JS, Greer JA, Muzikansky A, et al: Early palliative care for patients with metastatic non-small-cell lung cancer. N Engl J Med 363: , Quill TE, Abernethy AP: Generalist plus specialist palliative care: Creating a more sustainable model. N Engl J Med 368: , Meier DE, Sieger CE: A Guide to Building a Hospital Based Palliative Care Program. New York, NY, Center to Advance Palliative Care, Levy MH, Back A, Benedetti C, et al: NCCN clinical practice guidelines in oncology: Palliative care. J Natl Compr Canc Netw 7: , Glare PA, Semple D, Stabler SM, et al: Palliative care in the outpatient oncology setting: Evaluation of a practical set of referral criteria. J Oncol Pract 7: , Glare P, Plakovic K, Schloms A, et al: Study using the NCCN guidelines for palliative care to screen patients for palliative care needs and referral to palliative care specialists. J Natl Compr Canc Netw 11: , Last JM: A Dictionary of Epidemiology (ed 4). Oxford, United Kingdom, Oxford University Press, Seow H, Barbera L, Sutradhar R, et al: Trajectory of performance status and symptom scores for patients with cancer during the last 6 months of life. J Clin Oncol 29: , Chang VT, Hwang SS, Kasimis B, et al: Shorter symptom assessment instruments: The Condensed Memorial Symptom Assessment Scale (CMSAS). Cancer Invest 22: , Weissman DE, Meier DE: Identifying patients in need of a palliative care assessment in the hospital setting: A consensus report from the Center to Advance Palliative Care. J Palliat Med 14:17-23, Scandrett KG, Reitschuler-Cross EB, Nelson L, et al: Feasibility and effectiveness of the NEST13 as a screening tool for advanced illness care needs. J Palliat Med 13: , Morita T, Fujimoto K, Namba M, et al: Palliative care needs of cancer outpatients receiving chemotherapy: An audit of a clinical screening project. Support Care Cancer 16: , Adelson K, Paris J, Smith C, et al: Standardized criteria for required palliative care consultation on the solid tumor oncology service. Presented at the American Society of Clinical Oncology Quality Care Symposium, November 1, 2013 (abstr 37) 15. Barbera L, Seow H, Howell D, et al: Symptom burden and performance status in a population-based cohort of ambulatory cancer patients. Cancer 116: , Miller RJ: Predicting survival in the advanced cancer patient. Henry Ford Hosp Med J 39:81-84, Ferrieri P: Proceedings of the Jones criteria workshop. Circulation 106: , Osse BH, Vernooij-Dassen MJ, de Vree BP, et al: Assessment of the need for palliative care as perceived by individual cancer patients and their families: A review of instruments for improving patient participation in palliative care. Cancer 88: , Richardson A, Medina J, Brown V, et al: Patients needs assessment in cancer care: A review of assessment tools. Support Care Cancer 15: , 2007 Copyright 2014 by American Society of Clinical Oncology JANUARY 2015 jop.ascopubs.org e85

6 Glare and Chow Appendix Implications of Widespread Implementation of Screening Tool for Palliative Care Specialist Workforce If a score of 5 on this tool triggered oncologists to consider consultation with a palliative care (PC) specialist, there would be the potential for the volume of consults generated to have an adverse impact on the already-strained workforce. Estimating the volume of referrals requires a prediction of the likely prevalence of patients who fit these criteria nationally. The SEER data set is unlikely to help with generating this estimate, because the only one of the parameters in the screening tool that could be estimated using SEER data is the number of incident cases of locally advanced or metastatic cancer. Unfortunately, SEER does not include data on performance status (PS), symptoms, or comorbidity. Data from the province of Ontario, Canada, may be helpful, because it is now provincial policy that all ambulatory patients attending cancer centers (approximately 20% of all patients with cancer in Ontario) have their PS and symptom burden scored at each visit. 9,15 In Ontario, at their first visit to an oncologist, approximately 25% of patients have a palliative performance score of 60 or 70, which is equivalent to an Eastern Cooperative Oncology Group (ECOG) score of 2 (2 points). 15 Similar percentages of patients at new visits have poorly controlled physical symptoms and moderate-severe psychological distress (1 point each). 15 Presumably, for most of these patients, locally advanced or metastatic disease (2 points) rather than severe comorbidities (1 point) is the cause of their poor health. If it is assumed that half the patients with advanced disease and a poor PS also have uncontrolled symptoms or moderate-severe distress, approximately one in eight new visits will result in patients scoring 5 points on this tool, screening positive. Furthermore, in patients who subsequently died, at 6 months before death, the median ECOG PS was 2, and one third of patients had moderate-tosevere symptoms or distress. 9 Their PS deteriorated and the symptom burden rose as death approached, especially 2 to 3 months before death. This would result in 17% (one third of 50%) screening positive 6 months before death, and almost all patients screening positive 3 months before death. Extrapolating these data to the United States, where there are approximately 1.5 million new cases of cancer per year and half a million cancer deaths per year, approximately 187,500 patients would screen positive at first visit, and 85,000 (approximately half of them) would be expected to die in the next 6 months. Were the model of ambulatory palliative care developed for lung cancer at Massachusetts General Hospital to be adopted for all cancer types, 2 these patients would be seen in clinic monthly approximately four times for the half who died and 12 times for those who did not. Therefore, screening could generate up to 1.5 million physician-patient encounters per year, depending on the percentage of positive screens actually referred and the yield of screening (new referrals generated by screening process). Assuming a full-time equivalent (FTE) PC specialist may see 15 patients per day and works 260 days per year, it is estimated that another 400 FTEs would be needed to be employed for this purpose. The PC workforce is strained as it is, with approximately an 8,000 FTE physician shortage. The balance between appropriate referrals and a workforce that can meet the demand is truly the issue here and supports the call for a distinction between generalists and PC specialists. 3 e86 JOURNAL OF ONCOLOGY PRACTICE VOL. 11, ISSUE 1 Copyright 2014 by American Society of Clinical Oncology