Defining Patient Engagement in Research: A Qualitative Analysis ISPOR Patient Centered Special Interest Group Open Meeting ISPOR 20th Annual European Congress 7 November 2017 Glasgow, Scotland Panelists Rachel Harrington, Pre-Doctoral Fellow, Institute for Health Research and Policy / College of Pharmacy, University of Illinois at Chicago, Chicago, IL, USA Elisabeth M. Oehrlein, BA, PhD Candidate, Dept. of Pharmaceutical Health Services Research, University of Maryland School of Pharmacy, Baltimore, MD, USA Russell Wheeler, Patient Advocate for Leber's Hereditany Optic Neuropathy, Winchester, UK Rob Camp, Patient Advocate, EUPATI Spain and EURORDIS, Barcelona, Spain Robert N. McBurney, PhD, Co-Principal Investigator, iconquerms - the MS People-Powered Research Network, Accelerated Cure Project for MS, Waltham, MA, USA 2 1
Outline 1) Summary of systematic literature review and definition extraction 2) Qualitative analysis of definitions 3) Critique of definition quality 4) Synthesis and discussion 3 Summary of Systematic Literature Review & Definition Extraction Elisabeth M. Oehrlein 2
Systematic Literature Review Numerous definitions of patient engagement and patient centeredness Context is key What do these terms mean in the context of health economics and outcomes research? How can we be sure that we are engaging patients appropriately or that our research is patient-centered? Measurement 5 Identification Records identified through database searching on (n =1765) Additional records identified through other sources (n = 8 ) Systematic Literature Review Results & Definition Extraction Screening Eligibility Records screened (n = 1767 ) Full-text articles assessed for eligibility (n = 514) Records after duplicates removed PubMed/EMBASE (n = 1767 ) Records excluded (n = 1335 ) Full-text articles excluded, with reasons (n = 325) Included Definitions Extracted (n=242) Articles included (n = 276) Excluded for no code-able data (n=3) Analyzed Definitions Coded (n = 239) 3
Method for Definition Extraction Term Definition Reviewer 1 Reviewer 2 Type (original, adapted, adopted) Context Tiebreaker as needed Framework (original, adapted, adopted) 7 Characteristics of Sample Patient Focused (3) Patient Focused Drug Development (1) Patient Focused Outcomes Research (2) Patient Centered Outcomes Research (5) Patient Centered/Centric/Centeredness (80) Consumer Centered/Centric/Centeredness (0) Person/People Centered/Centric/Centeredness (7) Patient Engagement (28) Consumer Engagement (limit to health care) (1) Patient Empowerment (14) Patient Involvement (20) Consumer Involvement (0) Patient Input (1) Patient Oriented (1) Patient Participation (24) providing care that is respectful of, and responsive to, individual patient preferences, needs and values, and ensuring that patient values guide all clinical decisions. implies a greater ability and motivation to solve health-related problems, the exchange of relevant information with clinicians, shared decision making, the capacity to cope with complications, and follow through with treatment 8 4
Qualitative Analysis Methods & Results Rachel Harrington Motivating Questions What are the common characteristics of definitions related to the concept of patient engagement? Do these characteristics vary by definition context? By defined term? Are there particular concepts unique to engagement - in research? Based on stakeholder review, are there conceptual gaps in current definitions? 10 5
Qualitative Content Analysis (QCA) A method for the subjective interpretation of the content of text data through the systematic classification process of coding and identifying themes or patterns. (Hsieh 2005) Hypothesis generating, not hypothesis testing Semi-quantitatively assess qualitative (text) data Focus on the characteristics of language, accounting for the content and context of the text Strengths Systematic way of categorizing, summarizing, and relating content Limitations Strong focus on coding limits ability to portray richer understanding of context 11 QCA Process Definitions Microanalysis of Definitions (30% Random Sample) Develop Coding Dictionary Pilot-test Coding Dictionary, Revise, Complete Coding, Review Analysis Team-based approach The use of a coding dictionary is necessary to ensure consistency of approach and common interpretation. 12 6
Coding Dictionary How Characteristics Who Conductor, Target Definition Why Goals Where Setting When Stage of activity Fixed list of allowable terms for each code category If no term could be identified within a category, reviewers marked undetermined 13 Example of Coding Dictionary: How HOW High-Level Term Sub-Term(s) Example / Clarification access access: to care/services access: to information autonomy changing/evolving collaboration communication communication: accessible Presenting information/ content in lay vs. technical terms, approachable communication: effective communication: patient-provider communication: remove barriers communication: skills compassion How had the longest list of possible terms: 60 top level terms, 39 subterms Most other categories averaged 5-6 terms per list. 14 7
Example of Coding Spreadsheet Definition: depends on understanding how patients are engaged in their own treatment decisions in order to identify areas of unmet need. It also relies on study designs that best capture the endpoints and outcomes that are meaningful to patients. Fields for coding categories Select values from dropdown that best represent definition 15 Summary of Preliminary Results Presented today: Characteristics of Definitions (How) Setting of Definition (Where) Stage of Activity (When) What did we find? Areas of consistency: Characteristics across Searched Terms Areas of inconsistency Relative ranking ( importance ) of definition characteristics vary Stage of activity varies by setting Though these represent preliminary findings, trends are already emerging 16 8
Defined Terms n % Patient Centered 73 30.54 Patient Engagement 37 15.48 Patient Participation 32 13.39 Patient Empowerment 29 12.13 Patient Involvement 27 11.30 Patient Centeredness 19 7.95 Person Centered 11 4.60 Patient Focused 4 1.67 Patient Centric 3 1.26 Consumer Involvement 2 0.84 Patient Input 1 0.42 Patient Oriented 1 0.42 17 Heat Map Top 10 Characteristics by Defined Term Patient Rank Centered 1 patient: preferences 2 3 4 patient: perspective decisionmaking: shared relationship: patientprovider 5 respect 6 7 8 9 10 Patient Engagement process: active patient involvement participant collaboration Patient Participation participant decision making: shared patient involvement information sharing Patient Empowerment patient in control empowerment set of behaviors: self efficacy autonomy decisionmaking: shared decision making: partner clinical Defined Term Patient Patient Involvemnt Centeredness participant patient involvement process: active relationship: patient provider individualized approach: care person decisionmaking: shared understanding engagement Supporting / Support for individualized approach: care engagement process: active decision making collaboration patient: perspective patient involvement holistic approach to patient person partner patient partner relationship: patient provider information sharing consultation access: to care/services autonomy set of behaviors decision making decisionmaking: informed/comp etent influence participant patient education & skill development process: active decisionmaking decisionmaking: clinical /add ress context: psychosocial (bio) partner decisionmaking: collaboration informed/com petent Person Centered patient: perspective Patient Focused individualized approach: care individualized communication: approach: care patient provider holistic approach to patient patient: preferences partner collaboration decision making: shared holistic approach to patient patient: perspective partner Patient Centric engagement patient: perspective patient: preferences individualized approach information sharing outcomes: maximize benefit Consumer Involvement collaboration communication consultation Patient Input patient: perspective information sharing "informer" leader expert partner n/a Patient Oriented information sharing meaningful to patient relationship: patient provider n/a n/a understanding n/a n/a /add ress context: psychosocial patient comfort patient choice n/a n/a n/a (bio) participant respect Supporting / Support for process: adaptive/respon sive process: coordinated Supporting / Support for Take-away: There are a lot of similarities! patient in control process: adaptive/respon sive process: appropriate n/a n/a n/a n/a n/a n/a n/a n/a n/a 50% of Defined Terms 40% 49% of Defined Terms 30% 39% of Defined Terms 20% 29% of Defined Terms 10% 19% of Defined Terms 18 <10% of Defined Terms 9
Zooming In - Top 3 Terms Defined Term Rank Patient Centered (n = 73) Patient Engagement (n = 37) Patient Participation (n = 37) 1 patient: preferences process: active participant 2 patient: perspective patient involvement decision making: shared 3 decision making: shared participant patient involvement 4 relationship patient provider collaboration information sharing 5 respect decision making: shared decision making: clinical 6 individualized approach: care engagement process: active 7 patient involvement patient relationship: patient provider 8 holistic approach to patient partner autonomy 9 person set of behaviors decision making 10 partner decision making: informed/competent influence 19 Terms Appearing Most Often in Top 10 % Defined Concept Terms partner 58.3% patient: perspective 50.0% collaboration 41.7% decision-making: shared 33.3% individualized approach: care 33.3% participant 33.3% engagement 25.0% holistic approach to patient 25.0% patient: preferences 25.0% information sharing 25.0% patient involvement 25.0% process: active 25.0% Supporting / Support for 25.0% Example References to incorporating patient feelings, experiences, values Example all of patient, whole patient, beyond physical Example References to incorporating patient expectation, needs Example "taking action", "Active " Example "emotional support", "support patients in " 20 10
Top Characteristics, Overall & By Setting Overall Research Healthcare Other Undetermined n % n % Rank n % Rank n % Rank n % Rank Definitions, Overall 239 23 9.6% 173 72.4% 11 4.6% 32 13.4% Top Characteristics of Activity (1) participant 42 17.6% 4 17.4% #10 33 19.1% #1 2 18.2% #7 3 9.4% #11 (2) patient involvement 39 16.3% 9 39.1% #1 24 13.9% #5 4 36.4% #2 2 6.3% #22 (3) decision making: shared 36 15.1% 2 8.7% #14 24 13.9% #4 4 36.4% #1 6 18.8% #1 (4) patient: preferences 35 14.6% 2 8.7% #15 32 18.5% #3 0 0.0% 1 3.1% #34 (5) relationship: patient provider 35 14.6% 2 8.7% #22 33 19.1% #2 0 0.0% 0 0.0% (6) partner 31 13.0% 7 30.4% #2 18 10.4% #11 3 27.3% #3 3 9.4% #12 (7) patient: perspective 30 12.6% 4 17.4% #9 22 12.7% #7 0 0.0% 4 12.5% #3 (8) process: active 27 11.3% 5 21.7% #5 19 11.0% #9 3 27.3% #5 0 0.0% (9) individualized approach: care 26 10.9% 0 0.0% 23 13.3% #6 0 0.0% 3 9.4% #10 (10) information sharing 22 9.2% 2 8.7% #16 18 10.4% #10 0 0.0% 2 6.3% #18 Consistently high ranking for a participant (#1 overall), and general consistency across top 10 Terms vary the most among Research & Undetermined Settings Healthcare is most frequent setting Definitions referring to patient-level care made up 20.6% of this 21 Heat Map Top 10 Characteristics by Setting Setting Rank Research Healthcare Other Unidentified 1 patient involvement participant decision making: shared decision making: shared 2 partner relationship: patientprovider patient involvement patient in control 3 collaboration patient: preferences partner patient: perspective 4 equal decision making: shared patient in control understanding 5 process: active patient involvement process: active access: to information 6 process: engagement individualized approach: care collaboration autonomy 7 engagement patient: perspective participant collaboration 8 patient respect process communication 9 patient: perspective process: active representation/diversity 10 participant information sharing responsibility Appears consistently in 3 settings Appears consistently in 2 Settings Only appears in 1 setting decision making: informed/competent individualized approach: care 22 11
Stage at which Activity Occurs Overall & By Setting Overall Research Healthcare Other Undetermined n % n % Rank n % Rank n % Rank n % Rank Most Prevalent Stage of Activity 1. Strategy & Priority Setting 14 5.9% 3 13.0% #5 8 4.6% #4 1 9.1% #4* 2 6.3% #3* 2. Design & Planning 30 12.6% 9 39.1% #1 18 10.4% #2 2 18.2% #3 1 3.1% #4 3. Conduct & Operation 13 5.4% 4 17.4% #3* 7 4.0% #5 0 0.0% 2 6.3% #3* 4. Dissemination & Communication 6 2.5% 4 17.4% #3* 2 1.2% #7 0 0.0% 0 0.0% 5. In Practice / In Use 157 65.7% 4 17.4% #3* 142 82.1% #1 4 36.4% #2 7 21.9% #2 All 14 5.9% 8 34.8% #2 5 2.9% #6 1 9.1% #4* 0 0.0% Undetermined 43 18.0% 0 0.0% 18 10.4% #3 5 45.5% #1 20 62.5% #1 No consistent #1. Strategy and Priority Setting generally quite low. Among definitions in the Research setting, findings are reassuring, but not surprising: All is more frequent than in other settings Dissemination and communication higher than in Healthcare 23 What may all this tell us about a definition? Based on current definitions in the literature, is there a consensus on what defines Patient Engagement in Research? Active and collaborative involvement of patients across all stages of the research process, engaging in shared decisionmaking while recognizing and supporting patients as partners with individual preferences and perspectives Can we stop here? Is this adequate? Is it actionable? Measurable? Is this the Frankenstein s Monster of definitions? 24 12
Next Steps Finalize analysis Stakeholder review of Patient Engagement in Research concepts Draft manuscript for review by full ISPOR Patient- Centeredness SIG (over 400 review group members) and Patient Representatives Roundtables Including proposed definition in comparison/contrast to existing literature Final manuscript to Value in Health, 1Q 2018 25 Acknowledgements Analysis Coding Team Rob Camp Maya Hanna Rachel Harrington Francis Nguyen Amie Scott Russell Wheeler Data Management Assistance Hager Elgendi Shannon Marrow 26 13
Critique of Definitions Quality & Concepts Russell Wheeler and Rob Camp Motivation of Critique Issues: Alignment of objectives - Difficult for a virtual collaboration where participants have very different backgrounds and in many cases have never met each other. Harmonisation of outcomes - Given those constraints, it is not surprising that outputs from each Twin can vary a lot. Quality vs Purity - (or bias vs noise) A dichotomy where an attempt to weed out distracting definitions could result in bias 28 14
Underlying constraints Problem is pervasive; many attempts to resolve the lack of a definition of Patient Engagement but none are convincing. Clear need for a more scientific approach and widespread recognition that a well researched and robust definition is needed. Dictates a literature review as a starting point but what to do when the literature is both voluminous and flawed. Inevitable issues maintaining alignment of objectives under these circumstances 29 Quality vs Purity Inclusivity vs desire to focus on high quality definitions Volume of articles in which key terms are found is large almost 1,800 in total. Systematic mechanism in order to reduce bias and to make the task manageable. Large amounts of poor quality definitions make it through the process Any attempt to prevent this will introduce bias the bias vs noise dilemma is unavoidable. 30 15
Twinning Multiple reviewers necessary to reduce bias Surprisingly large variance between twins leading to a process where inevitably too much chaff is left in We return to the alignment of objectives how exactly have we managed to have so much diversity within what we would have expected to have been a group of like-minded people? 31 It s all a question of perspective 32 16
Discussion Robert McBurney Impressions The team took on an enormous task and generated very important findings (ISPOR support essential) Patient as Participant as top theme Strategy & Priority Setting of low importance??? All stakeholders will benefit from fit for purposes terms with definitions that facilitate communication 34 17
Questions Will the first article contain recommended terms with definitions that capture the learnings? Beyond the publication(s), what is the path to impact for the findings from this excellent work in healthcare? in research and decision making? How can ISPOR lead on the path to impact? What recommendations would you share with other initiatives? 35 Sign up as a Review Group Member Join ISPOR Special Interest Groups Need ISPOR membership number Give business card to ISPOR staff or e-mail sigs@ispor.org Submit your evaluation using the ISPOR app 18
QUESTIONS? Thank You 19