Indicators of Integration of Oncology and Palliative Care Programs: An International Consensus

Transcription

1 Annals of Oncology Advance Access published June 18, Indicators of Integration of Oncology and Palliative Care Programs: An International Consensus D. Hui 1, S. Bansal 1, F. Strasser 2, T. Morita 3, A. Caraceni 4, M. Davis 5, N. Cherny 6, S. Kaasa 7, D. Currow 8, A. Abernethy 9, C. Nekolaichuk 10, E. Bruera 1 1 Department of Palliative Care and Rehabilitation Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX, USA 2 Oncological Palliative Medicine, Hematology-Oncology, Cantonal Hospital, St. Gallen, Switzerland 3 Department of Palliative and Supportive Care and Seirei Hospice, Seirei Mikatahara General Hospital, Hamamatsu, Japan 4 Palliative Care, Pain Therapy and Rehabilitation, Fondazione IRCCS Istituto Nazionale dei Tumori di Milano, Milan, Italy 5 Department of Solid Tumor Oncology, Taussig Cancer Institute, Cleveland, Ohio, USA 6 Cancer Pain and Palliative Medicine Service, Department of Medical Oncology, Shaare Zedek Medical Center, Jerusalem, Israel 7 European Palliative Care Research Centre, Department of Cancer Research and Molecular Medicine, Norwegian University of Science and Technology and The Cancer Clinic, St. Olavs Hospital Trondheim University Hospital, Trondheim, Norway 8 Palliative and Supportive Services, Flinders University, Adelaide, South Australia 9 Duke Clinical Research Institute, Duke University School of Medicine, Durham, North Carolina, USA 10 Division of Palliative Care Medicine, Department of Oncology, University of Alberta, Edmonton, Canada Corresponding author: Dr. David Hui, Department of Palliative Care and Rehabilitation Medicine, Unit 1414, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Boulevard, Houston, TX, USA 77030, Phone: , Fax: , dhui@mdanderson.org The Author Published by Oxford University Press on behalf of the European Society for Medical Oncology. All rights reserved. For permissions, please journals.permissions@oup.com.

2 2 Abstract Background: Recently, the concept of integrating oncology and palliative care has gained wide professional and scientific support; however, a global consensus on what constitutes integration is unavailable. We conducted a Delphi Survey to develop a consensus list of indicators of integration of specialty palliative care and oncology programs for advanced cancer patients in hospitals with 100 beds. Design: International experts on integration rated a list of indicators on integration over 3 iterative rounds under 5 categories: clinical structure, processes, outcomes, education and research. Consensus was defined a priori by an agreement of 70%. Major criteria (i.e. most relevant and important indicators) were subsequently identified. Results: Among 47 experts surveyed, 46 (98%), 45 (96%) and 45 (96%) responded over the 3 rounds. 19 (40%) were female, 24 (51%) were from North America and 14 (30%) were from Europe. 16 (34%), 7 (15%) and 25 (53%) practiced palliative care, oncology and both specialties, respectively. After 3 rounds of deliberation, the panelists reached consensus on 13 major and 30 minor indicators. Major indicators included 2 related to structure (consensus 95-98%), 4 on processes (88-98%), 3 on outcomes (88-91%) and 4 on education (93-100%). The major indicators were considered to be clearly stated (9.8/10), objective (9.4/10), amenable to accurate coding (9.5/10) and applicable to their own countries (9.4/10). Conclusions: Our international experts reached broad consensus on a list of indicators of integration, which may be used to identify centers with a high level of integration, and facilitate benchmarking, quality improvement and research. Key words: access, indicators, integration, health systems, neoplasms, palliative care Key Message: "In this study, international experts reached consensus on 13 major and 30 minor indicators of integration of specialty palliative care and oncology programs for advanced cancer patients. These indicators may be used to identify centers with a high level of integration, and facilitate benchmarking, quality improvement and research."

3 3 Introduction Recognizing the significant supportive care needs among cancer patients and the role that palliative care plays in addressing these needs,[1-4] multiple national and international professional organizations have called for increased integration of oncology and palliative care.[5-8] Since 2004, the European Society for Medical Oncology (EMSO) offered an incentive program of ESMO designated centers of integrated oncology and palliative care based on 13 criteria that focused on the palliative care related processes, education and research; however, these criteria did not explicitly include formal clinical structures.[6] To date, a global consensus on what constitutes an acceptable level of integration of oncology and palliative care is currently lacking. To identify the various aspects of integration, we recently conducted a systematic review and documented 38 potential indicators spanning the clinical, education, research and administrative aspects of integration.[9] Because integration is a complex and ill-defined concept, it has been unclear whether a consensus could be reached regarding a list of indicators on integration that is applicable internationally. The availability of such indicators would allow (1) policy makers and administrators to set goals for their programs, compare their hospital s performance, standardize the care provided, and develop quality improvement projects; (2) patients and referring physicians to recognize hospitals that offer integrated palliative care services; (3) organizations to provide designations for cancer centers and hospitals that have achieved a certain level of integration; and (4) researchers to study the outcomes associated with integration. In this Delphi study of international experts, we aimed to develop a consensus list of indicators of integration of palliative care and oncology programs for advanced cancer patients in hospitals with 100 beds.

4 4 Methods Participants To be eligible for this Delphi study, participants had to meet all 3 criteria: (1) A physician with an active (at least 20% clinical) specialty clinical practice in either palliative care and/or oncology, with at least 5 years of post-qualification clinical experience; (2) at least one of the following: board certification or equivalent in both oncology and palliative care, have published in the area of integration of palliative care and oncology in the last 10 years, have been involved in national/international palliative care guideline development on the topic of integration; and (3) able to communicate in written English. These highly stringent criteria were designed to select a group of international experts who were active clinically and had an excellent understanding of integration through their training, practice and/or research. Following the Delphi methodology, we identified potentially eligible candidates based on a literature review on integration, membership in the Multinational Association of Supportive Care in Cancer (MASCC) Palliative Care study group and the ESMO Palliative Care Working Group, and nominations by the steering committee. Potentially eligible candidates were sent an invitation outlining the eligibility criteria above and the study process. To maximize inclusion, they were also requested to forward the names of other potential candidates, who were also subsequently contacted for this study. All individuals who responded, confirmed they met the eligibility criteria and expressed a willingness to participate were included in this Delphi study. This study was reviewed by the institutional review board at MD Anderson Cancer Center, which approved it with wavier of informed consent. List of Potential Indicators Because indicators for integration could vary widely based on the clinical setting (large vs. small hospitals) and who is delivering it (oncologists vs. palliative care teams), this study specifically focused on indicators of integration of specialty palliative care programs and oncology programs for advanced cancer patients in hospitals with 100

5 5 beds. The list of potential indicators was generated based on a recent systematic review conducted by our group that identified 38 indicators on integration of oncology and palliative care[9] and a conceptual framework on integrated palliative care.[10] This was further supplemented by a review of outcome measures/indicators that assessed the quality of palliative care delivery from various organizations and investigators, including the ASCO Quality Oncology Practice Initiative (QOPI), the ESMO criteria for designated centers of integrated oncology and palliative care, the National Consensus Project, and the National Quality Forum.[6, 11-14] Members of the steering committee provided input to fine tune the list of indicators and the wording to be included prior to each Delphi round. Because the arrangements for hospice services and advance care planning vary by geographic regions, indicators on these topics were qualified by the phase: if applicable in your country. We classified all potential clinical indicators under the Donabedian triad of structure, processes and outcomes, and also included education and research as two separate categories. Survey Process Qualtrics Online Survey (Qualtrics, Utah, USA) was used to conduct the anonymous web-based surveys between September 8, 2014 and February 16, Our Delphi study consisted of 3 rounds, each lasting 4 weeks and separated 4 weeks apart. Nonrespondents were sent weekly reminders. No financial incentives were provided. The first Delphi round examined 52 putative indicators. We asked the experts to independently state their level of agreement using a 5-point Likert scale ( strongly agree, agree, neither agree nor disagree, disagree and strongly disagree ) for each indicator whether they feel it should be used as an indicator of integration of palliative care programs and oncology programs in mid- to large- size hospitals (100 beds or more). Strongly agree and agree were combined and coded as agreement in our analysis. A consensus in this study was defined a priori as an agreement (i.e. agree or strongly agree) by a minimum of 70% of the experts. We also collected basic demographic information from the experts, such as age group, type of clinical practice, and years of experience.

6 6 The second Delphi round included 60 putative indicators, with 8 new indicators added based on the comments of participants from the first round. For each indicator, participants were provided with the average percentage of agreement in the previous round, and asked to state their level of agreement in light of the group s opinion. Furthermore, they were asked to rate the relative importance of each indicator that they agreed to be useful (from 0 to 10, where 0=least important, 10=most important). The third Delphi round focused on a shortlist of major criteria, defined as the most relevant and important indicators related to integration. All criteria with >90% agreement from the second Delphi round were presented again to the Delphi panel to decide if they should individually be considered as a major indicator. Any criteria that reached a consensus in the second round and not included as a major indicator after the third round were considered as minor indicators. To further refine the definition of the indicators, we asked our panelists to rate their agreement on the makeup of the interdisciplinary team members ( consisting at least of (1) physician, (2) nurse, (3) social worker, and (4) pastoral care or other counselor (adapted from US hospice criteria) ; consisting at least of (1) physician, (2) nurse, and (3) social worker, pastoral care or other counselor ), the criteria for early referral ( within 1 month of diagnosis of advanced cancer ; within 2 months of diagnosis of advanced cancer ; within 3 months of diagnosis of advanced cancer ) and the duration of mandatory palliative care rotations for oncology fellows ( a minimum of 1 month rotation in palliative care ). The experts were then asked to rate the major criteria using a 0 (not at all) to 10 (best) numeric rating scale in regard to the following: (1) clearly stated, (2) objective, (3) amenable to accurate coding, and (4) applicable to their own countries. Statistical Analysis We used descriptive statistics, such as mean, standard deviations, median, interquartile range, and proportions to summarize the data. We used the Wilcoxon rank sum test to determine stability of responses between the first and second Delphi rounds. A P-value of less than 0.05 was considered to be statistically significant. Statistical analysis was conducted with STATA version 12.1 (College Station, TX, USA).

7 7 Results Response rates and participant characteristics Among 47 experts surveyed, 46 (98%), 45 (96%) and 45 (96%) responded over the 3 rounds. The participants characteristics are shown in Table 1. First Delphi round Our panelists reached consensus on 40/52 (77%) indicators (Table 2). The participants also suggested 8 new indicators that were included in the second round. Second Delphi round 43/60 (72%) indicators reached the consensus threshold (Table 2). 22 (37%) indicators had an average importance of 8/10. The average level of importance highly correlated with the percentage of agreement (rho=0.91). We found no statistically significant differences in the level of agreement between the first and second round for a vast majority of the indicators, suggesting that the findings were stable (Table 2). The only two indicators with a significant change had agreement of 70% in both rounds and both decreased further in the second round. Third Delphi round During the first and second round, some participants commented if a distinction should be made between major and minor criteria: I wonder what is needed for a minimum level of integration, and what is nice to have. Thus, for the third round, the steering committee identified 13 indicators that had over 90% consensus in the second round, and asked the panel to decide if they should be considered as major indicators. As shown in Table 3, all 13 indicators had an agreement between 88% and 100%. The remaining 30 indicators with consensus between 70% and 90% in the second round were considered as minor indicators (Table 2). Regarding the major indicators, 36 (80%) participants agreed that the interdisciplinary team must include at least the following disciplines: (1) physician, (2) nurse, (3) social

8 8 worker, and (4) pastoral care or other counselor, which was adapted from US hospice criteria. Interestingly, 34 (76%) agreed on a less stringent criteria of at least a (1) physician, (2) nurse, and (3) social worker, pastoral care or other counselor. In regard to the timing of palliative care referral, only 24 (53%), 19 (42%) and 21 (47%) panelists agreed that referral should be within 3 months, 2 months and 1 month of the diagnosis of advanced cancer, respectively. Finally, 39 (87%) endorsed that routine rotation in palliative care for oncology fellows should be a minimum of 1 month. The panelists considered these major indicators to be clearly stated (9.8/10), objective (9.4/10), amenable to accurate coding (9.5/10) and applicable to their own countries (9.4/10). Discussion Our international experts reached broad consensus on a list of system indicators of integration of palliative care and oncology programs for advanced cancer patients in hospitals with 100 beds. After 3 rounds, they identified 13 major indicators (91-100% agreement in second round, % agreement in third round) and 30 minor indicators (70-90% agreement in second round). With further validation, these indicators may be used to identify centers with a high level of integration, and facilitate benchmarking, quality improvement and research. To our knowledge, this is the first study to specifically examine system indicators of integration of oncology and palliative care. Our findings were based on the consensus of a cohort of international experts with significant clinical experience and diverse professional backgrounds, which increased its generalizability. Furthermore, we had a high participation rate over 3 rounds. There was minimal shift in opinion between the first and second round, suggesting that our findings were robust. Two structural elements were considered as major indicators of integration: the presence of palliative care inpatient consultation team and outpatient clinic. Indeed, both services have important purposes; inpatient services remain the backbone of

9 9 palliative care, providing symptom management and discharge planning for acutely ill patients. In contrast, outpatient clinics facilitate early palliative care access for cancer patients, which allows for preventative symptom control measures, longitudinal education and support, and advance care planning. In a national survey conducted in 2010, we found that palliative care inpatient service were available in 92% of National Cancer Institute (NCI) designated cancer centers and 56% of non-nci designated cancer centers; however, outpatient clinics were present in 59% and 22%, respectively.[15] Interestingly, acute palliative care units and community-based palliative care services did not reach the threshold for consensus as indicators of integration, although they serve important purposes within the continuum of care.[16] This may partly be related to the observation that few Delphi participants worked in these two settings. Four major indicators on clinical processes were identified: interdisciplinary team, routine symptom screening, routine documentation of advance care planning, and early referral. Our panel also reached a consensus on the minimum requirement of an interdisciplinary team, with at least a (1) physician, (2) nurse, and (3) psychosocial team member. The other 3 process indicators were related to timely preventative interventions. The presence of routine symptom screening in the oncology setting was important because it would allow early symptom detection and treatment, and facilitate referral to specialty palliative care.[17] Several studies are ongoing to determine the outcomes associated with routine symptom screening followed by palliative care interventions. Advance care planning, end-of-life discussions and early palliative care involvement have been shown to reduce aggressive care at the end-of-life.[1, 18, 19] Although randomized controlled trials introduced palliative care within 1-3 months of advanced cancer diagnosis, only half of our panelists agreed with using this as a cutoff to define early palliative care.[2, 20] In metastatic diseases in which median survival may be as long as several years, the exact timing of early palliative care may be a factor of need rather than any specific time point from diagnosis. Further research is needed to identify what referral criteria are ideal.

10 10 Among the 3 major indicators related to clinical outcomes, 2 were established indicators already adopted by ASCO QOPI to examine the quality of end-of-life care.[11, 21] The third indicator Place of death consistent with patient s preference was proposed by a panelist during the first round. In contrast to non-hospital death which received only 66% agreement, this indicator is more personalized because it recognizes that patients and families may have different end-of-life care needs.[22] However, it would require that patients wishes be clearly stated. The remaining 4 major indicators were all related to education, highlighting the importance of this aspect in integration.[23] Specifically, oncology fellows should have didactic lectures, mandatory palliative care rotations, and combined educational activities with palliative care fellows. Buss et al. conducted a survey of oncology fellows in the US, and found that there were some knowledge gaps.[24] Furthermore, currently only approximately 25% of fellowship programs in the United States have mandatory palliative care rotations, and palliative care content is not a criteria for training program accreditation.[15, 24] On a positive side, the Global Core Curriculum for oncology training, which is endorsed by both ASCO and ESMO, emphasizes a wide range of important palliative care skills. Program directors and accreditation organizations have an important role to support the integration process. This study has several limitations. First, although the number of experts included in this study was large by Delphi standards, some countries and continents outside of North America and Europe were under-represented. Indeed, it was challenging to identify experts who fit our stringent criteria in those regions, partly because palliative care remained under-developed. Further studies are needed to examine indicators of integration in low resource countries. Second, we only surveyed physicians in this study. Other disciplines and patients may have different perspectives that would need to be explored in future studies. Third, because indicators may vary significantly by hospital size and resources, we decided to limit the indicators to mid- to large-size hospitals with at least 100 beds. Our expert panel appropriately consisted of individuals working predominantly in the acute care and academic settings. Thus, our results are

11 11 particularly relevant to integration of palliative care within tertiary care hospitals and cancer centers. More attention should be paid in future studies to integration at the community level, and how oncology professionals can deliver optimal palliative care. In summary, this Delphi study identified 13 major and 30 minor indicators of integration through consensus among international experts. At the institution level, these indicators represent milestones toward the vision of integration. Implementation of processes to achieve these milestones is likely going to be a stepwise process, requiring longitudinal commitment from oncologists, palliative care specialists, educators, and hospital administrators. In addition, successful integration will require proper funding, planning, documentation and quality improvement. Although palliative care has been found to be associated with decreased cost of end-of-life care, further studies are needed to identify the cost-effectiveness of instituting these measures. Acknowledgement: We would like to thank all international experts who participated in this Delphi study and provided valuable input. Funding: This research is supported in part by a University of Texas MD Anderson Cancer Center Support Grant (CA ). This study is also partly supported by research fund from the Multinational Association for Supportive Care in Cancer. D.H. is supported in part by a National Institutes of Health grant (R21CA A1) and an American Cancer Society Mentored Research Scholar Grant in Applied and Clinical Research (MRSG CCE).The funding organizations had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. Disclosure: No potential conflict of interest.

12 12 References 1. Hui D, Kim SH, Roquemore J et al. Impact of timing and setting of palliative care referral on quality of end-of-life care in cancer patients. Cancer 2014; 120: Temel JS, Greer JA, Muzikansky A et al. Early palliative care for patients with metastatic non-small-cell lung cancer. New England Journal of Medicine 2010; 363: Zimmermann C, Swami N, Krzyzanowska M et al. Early palliative care for patients with advanced cancer: a cluster-randomised controlled trial. Lancet 2014; 383: Abernethy AP, Currow DC, Shelby-James T et al. Delivery strategies to optimize resource utilization and performance status for patients with advanced life-limiting illness: results from the "palliative care trial" [ISRCTN ]. J Pain Symptom Manage 2013; 45: Ferris FD, Bruera E, Cherny N et al. Palliative cancer care a decade later: accomplishments, the need, next steps. Journal of Clinical Oncology 2009; 27: Cherny N, Catane R, Schrijvers D et al. European Society for Medical Oncology (ESMO) program for the integration of oncology and palliative care: A 5-year review of the designated centers' incentive program. Annals of Oncology 2010; 21: Kaasa S. Integration of general oncology and palliative care. Lancet Oncology 2013; 14: Saarto T. Palliative care and oncology: the case of early integration Eur J Pall Care 2014; 21: Hui D, Kim YJ, Park JC et al. Integration of Oncology and Palliative Care: A Systematic Review. Oncologist 2015; 20:

13 Bainbridge D, Brazil K, Krueger P et al. A proposed systems approach to the evaluation of integrated palliative care. BMC Palliat Care 2010; 9: McNiff KK, Neuss MN, Jacobson JO et al. Measuring supportive care in medical oncology practice: lessons learned from the quality oncology practice initiative. Journal of Clinical Oncology 2008; 26: National Consensus Project for Quality Palliative C. Clinical Practice Guidelines for Quality Palliative Care, Second Edition. In Cancer Endorsement Maintenance. In. Washington, USA: National Quality Forum Kamal AH, Gradison M, Maguire JM et al. Quality measures for palliative care in patients with cancer: a systematic review. J Oncol Pract 2014; 10: Hui D, Elsayem A, De la Cruz M et al. Availability and integration of palliative care at US cancer centers. JAMA 2010; 303: Davis MP, Strasser F, Cherny N. How well is palliative care integrated into cancer care? A MASCC, ESMO, and EAPC Project. Support Care Cancer Blum D, Rosa D, dewolf-linder S et al. Development and validation of a medical chart review checklist for symptom management performance of oncologists in the routine care of patients with advanced cancer. J Pain Symptom Manage 2014; 48: Mack JW, Cronin A, Keating NL et al. Associations between end-of-life discussion characteristics and care received near death: a prospective cohort study. J Clin Oncol 2012; 30: Wright AA, Zhang B, Ray A et al. Associations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA 2008; 300:

14 Bakitas M, Lyons KD, Hegel MT et al. The project ENABLE II randomized controlled trial to improve palliative care for rural patients with advanced cancer: baseline findings, methodological challenges, and solutions. Palliat Support Care 2009; 7: Earle CC, Landrum MB, Souza JM et al. Aggressiveness of cancer care near the end of life: is it a quality-of-care issue? Journal of Clinical Oncology 2008; 26: Agar M, Currow DC, Shelby-James TM et al. Preference for place of care and place of death in palliative care: are these different questions? Palliat Med 2008; 22: Abrahm JL. Integrating palliative care into comprehensive cancer care. JNCCN Journal of the National Comprehensive Cancer Network 2012; 10: Buss MK, Lessen DS, Sullivan AM et al. Hematology/oncology fellows' training in palliative care: results of a national survey. Cancer 2011; 117:

15 Table 1. Characteristics of Delphi Participants (N=47) Characteristics N (%)* Age group (17) (26) (47) (11) Female sex 19 (40) Continent North America 24 (51) Europe 14 (30) Asia 5 (11) Australia, Africa and South America 4 (9) Training Oncology only 7 (15) Palliative care only 15 (32) Palliative care and oncology 25 (53) Practice type Academic medicine 30 (65) Clinical practice 16 (35) Practice setting Tertiary care cancer center 33 (72) Tertiary care general hospital 13 (28) Community based cancer center 3 (7) Community based general hospital 4 (9) Others 2 (4) Years of oncology experience, median (Q1-Q3) 10 (7-22) Years of palliative care experience, median (Q1-Q3) 13 (8-21) * unless otherwise specified Among those who had oncology training Among those who had palliative care training

16 Table 2. Putative Indicators of Integration of Oncology and Palliative Care: First Two Rounds* Indicator Clinical Structure First round agreeme nt N (%) Second round agreeme nt N (%) Second round importa nce (0-10) P-value Categor y Presence of palliative care inpatient consultation team 44 (96) 44 (98) MAJOR Presence of palliative care outpatient clinic 42 (91) 41 (93) MAJOR Presence of dedicated palliative care beds 32 (71) 30 (67) Exclude Presence of community based palliative care as an outreach operated by the hospital Clinical Processes Presence of interdisciplinary palliative care team 44 (96) 45 (100) 24 (55) 17 (38) Exclude MAJOR Routine symptom screening in the outpatient oncology clinic 42 (93) 43 (96) MAJOR Institutionally accepted palliative care symptom management guidelines in written format Institutionally accepted palliative care referral criteria available in written format Institutionally accepted clinical care pathways (automatic triggers) for palliative care referral available Palliative care team routinely involved in multidisciplinary tumor conference for patient case discussions Presence of dually trained oncologist-palliative care specialists institution in the 38 (84) 36 (82) Minor 36 (78) 32 (74) Minor 38 (83) 31 (70) Minor 37 (80) 34 (76) Minor 26 (58) 22 (50) Exclude Oncologist: palliative care specialist ratio 20 (44) 9 (20) Exclude Presence of palliative care specialists among cancer center senior leadership (e.g. head of oncology department/division, chief executives) Routine documentation of advance care plans in patients with advanced cancer 37 (80) 39 (89) Minor 38 (83) 43 (98) MAJOR Early referral to palliative care 39 (85) 42 (95) NA 0.18 MAJOR Administration of systemic cancer therapy (e.g. chemotherapy, targeted agents) in palliative care patients possible 35 (78) 34 (76) Minor Palliative care clinics embedded in oncology clinics 30 (65) 25 (58) Exclude Proportion of patients with advanced cancer who had documentation of prognostic discussion NA 36 (80) 8.0 NA Minor Availability of same day inpatient palliative care consultation upon request NA 36 (82) 8.6 NA Minor Availability of same day outpatient palliative care consultation upon request NA 34 (77) 8.1 NA Minor Proportion of patients with advanced cancer who had quality of life assessed using a validated questionnaire at the time of diagnosis and then monthly Proportion of patients with advanced cancer who had satisfaction in relation to palliative care issues assessed (i.e. symptom management, information needs) at the time of diagnosis and then monthly Clinical Outcomes NA 23 (52) 6.8 NA Exclude NA 27 (60) 7.0 NA Exclude Proportion of advanced cancer patients seen by palliative care before death 41 (89) 38 (86) Minor Ratio of outpatient to inpatient palliative care referral 24 (53) 12 (27) Exclude Median time from diagnosis of advanced cancer to outpatient palliative care consultation Median time from diagnosis of advanced cancer to inpatient palliative care consultation 32 (70) 30 (67) Exclude 20 (43) 17 (38) Exclude Median time from outpatient palliative care consultation to death 32 (70) 23 (51) Exclude Median time from inpatient palliative care consultation to death 26 (58) 18 (40) Exclude Proportion of outpatients with pain ASSESSED on either of the last two visits before death 36 (78) 40 (91) MAJOR

17 Proportion of outpatients with pain intensity quantified on either of the last 2 visits before death Proportion of outpatients with plan of care for pain documented on either of the last 2 visits before death Proportion of outpatients with dyspnea ASSESSED on either of the last 2 visits before death Proportion of outpatients with dyspnea ADDRESSED on either of the last 2 visits before death 36 (78) 37 (86) Minor 32 (71) 38 (84) Minor 28 (61) 33 (73) Minor 29 (64) 32 (71) Minor Proportion of outpatients with any hospice enrollment 38 (84) 33 (73) Minor Proportion of patients with hospice enrollment within 3 days of death 37 (80) 34 (76) Minor Proportion of patients with hospice enrollment within 1 week of death 38 (83) 25 (58) Exclude For patients not referred, the proportion who had hospice or palliative care discussed within the last 2 months of life Proportion of patients with chemotherapy administered within the last 2 weeks of life 34 (76) 34 (76) Minor 34 (74) 38 (86) Minor Proportion of patients with intensive care unit admission in last 30 days of life 35 (76) 38 (86) Minor Proportion of patients with 2 or more emergency room visit in last 30 days of life 40 (87) 40 (91) MAJOR Proportion of patients with 2 or more hospital admission in last 30 days of life 37 (80) 37 (86) Minor Proportion of patients who died in a hospital 37 (80) 29 (66) Exclude Proportion of patients who died in an intensive care unit 37 (80) 38 (86) Minor Place of death consistent with patient s preference NA 41 (91) 8.6 NA MAJOR Education Didactic palliative care curriculum for oncology fellows 45 (98) 42 (95) MAJOR Continuing medical education in palliative care for attending oncologists 44 (96) 42 (93) MAJOR Combined palliative care and oncology educational activities for fellows/trainees 42 (91) 41 (91) MAJOR Oncology fellows have routine rotation in palliative care 43 (93) 42 (93) MAJOR Palliative care fellows have routine rotation in oncology 35 (78) 37 (82) Minor Continuing medical education in oncology for palliative care specialists NA 38 (84) 8.2 NA Minor Research Institutional funding for palliative oncology research 40 (87) 37 (82) Minor Peer reviewed publications in palliative oncology 37 (80) 39 (87) Minor Tenured faculty in palliative care 36 (78) 39 (87) Minor Endowed chair in palliative care 30 (65) 24 (53) Exclude Collaborative research between oncology and palliative care NA 39 (87) 7.9 NA Minor * In addition to the above indicators, we also included some general indicators. These included Presence of a comprehensive palliative care program (agreement in first and second round 85% and 91%, respectively), High proportion of patients referred to palliative care (agreement 52% and 34%), Improved patient care outcomes such as symptom management, quality of life, and quality of end-of-life care (agreement 87% and 91%), High level of collaboration between oncology and palliative care in educational activities (agreement 78% and 82%),and High level of collaboration between oncology and palliative care in research activities (agreement 67% and 57%). Since the level of agreement for these general indicators were highly consistent with specific indicators within each category, they were not included in the third round. We used the Wilcoxon rank sum test to determine stability of responses between the first and second Delphi rounds. Based on findings in the second round, major indicators had consensus >=90%, minor indicators had consensus 70-90%, and the others were excluded. Abbreviations: NA, not applicable

18 Table 3. Major Indicators of integration of palliative care and oncology programs for advanced cancer patients in hospitals with 100 beds after the third Delphi round Indicator Category Agreement N (%) 1. Presence of palliative care inpatient consultation team Structure 42 (98) 2. Presence of palliative care outpatient clinic Structure 41 (95) 3. Presence of interdisciplinary palliative care team* Process 42 (98) 4. Routine symptom screening in the outpatient oncology clinic Process 42 (98) 5. Routine documentation of advance care plans in patients with advanced Process 38 (88) cancer 6. Early referral to palliative care Process 39 (91) 7. Proportion of outpatients with pain assessed on either of the last two visits Outcome 38 (88) before death 8. Proportion of patients with 2 or more emergency room visits in last 30 days Outcome 37 (88) of life (negative indicator) 9. Place of death consistent with patient s preference Outcome 39 (91) 10. Didactic palliative care curriculum for oncology fellows provided by Education 43 (100) palliative care teams 11. Continuing medical education in palliative care for attending oncologists Education 42 (98) 12. Combined palliative care and oncology educational activities for Education 42 (98) fellows/trainees 13. Oncology fellows have routine rotation in palliative care Education 40 (93) * 36 (80%) participants agreed that the interdisciplinary team must include at least the following disciplines: (1) physician, (2) nurse, (3) social worker, and (4) pastoral care or other counselor, which was adapted from US hospice criteria. Interestingly, 34 (76%) agreed on a less stringent criteria of at least a (1) physician, (2) nurse, and (3) social worker, pastoral care or other counselor. Only 24 (53%), 19 (42%) and 21 (47%) panelists agreed that referral should be within 3 months, 2 months and 1 month of the diagnosis of advanced cancer, respectively. 39 (87%) endorsed that routine rotation in palliative care for oncology fellows should be a minimum of 1 month.