Insights and Innovation in Improving Health CRISP Seminar Series.

Transcription

1 Insights and Innovation in Improving Health CRISP Seminar Series 1

2 Definition of terms Current and recent use of mixed methods in health services research Strengths of qualitative vs. quantitative data Three dimensions of methodological combination Five mixed method designs with particular relevance for D/I research Common barriers to using mixed method approaches Suggestions for better integrating the publication of mixed method research 2

3 Several new terms to describe research previously classified as health services or outcomes research Pragmatic trials are designed to determine the effects of an intervention under the usual conditions in which it will be applied Vs. the highly controlled conditions of a classical efficacy trial 3

4 Implementation research Scientific investigations that support movement of evidence-based, effective health care approaches from the clinical knowledge base into routine use (Rubenstein and Pugh, 2006) Dissemination research The targeted distribution of information and intervention materials to a specific public health or clinical practice audience (Glasgow et al., 2012) 4

5 Employing mixed methods should be common practice in pragmatic trials and D/I research Understanding context is imperative Must understand the why behind the success or failure of intervention implementation Mixed methods are particularly helpful in the quest to understand context Generate data that are deep as well as broad 5

6 Recent increase in studies using mixed methods in health services and primary care research Particularly prevention-oriented studies: Weight loss and active living Immunizations Cancer screening Osteoporosis Cardiovascular health Substance abuse prevention 6

7 But still too few studies to help researchers interpret and understand the significant heterogeneity of trial results Only 47 of 1,651 empirical articles published in four top-ranked health services journals used mixed methods (Wisdom et al., 2012) Reporting of key methodological components, especially qualitative methods, remains uneven 7

8 Emphasis on deduction, objectivity, and generalization is useful when measuring intervention and/or implementation outcomes Involves testing and confirming hypotheses based on existing conceptual model and then obtaining a breadth, rather than a depth, of understanding of the predictors of successful implementation 8

9 Inductive, contextual approach Useful when seeking to address aspects that are intensive, nonlinear, or may require interaction (e.g., study of processes) Useful for eliciting perspectives, values, and opinions of stakeholders, participants, or consumers Useful for understanding why evidence-based practices are un/successfully implemented Useful for identifying strategies for facilitating implementation 9

10 Quantitative data to study outcomes; qualitative data to study processes Quantitative data to measure aspects of content; qualitative data to understand the context Qualitative data to explore a phenomenon and generate a conceptual model and hypotheses; quantitative data to test the hypotheses to confirm the validity of the model 10

11 Key to strong mixed methods research lies in the effective integration Combining and capitalizing on different methodological strengths increases both breadth and depth of understanding Effective integration is more than just collecting and then reporting two (or more) sets of results 11

12 Data must be mixed Clarity about implications of choices re: the nature and timing of the integration is imperative The underlying logic of mixing is that neither quantitative nor qualitative methods are sufficient in themselves to capture the trends and details of the situation. When used in combination, both quantitative and qualitative data yield a more complete analysis, and they complement each other. (Creswell et al. 2004) 12

13 Merging the data Two types of data are brought together to answer the same question or related questions Connecting the data Analysis of one set of data leads to the need for and subsequent collection of another set of data Embedding the data When qualitative studies of process or context are embedded within larger quantitative studies of outcomes to obtain depth or expansion 13

14 In both study design and analysis, researchers must understand (and be able to make explicit) the rationale for Order and sequencing Priority Purpose For each of the methods used For the particular combination 14

15 Methods may be simultaneous or sequential Expansion or explanation = Using one method to answer questions that emerged following data collection using another method Development = Using qualitative methods to generate initial data that will enable the effective use of other methods Sampling = Using one method to identify a sample of participants that will be used by another method 15

16 One method is typically operationalized as the dominant or primary method Most D/I studies have treated quantitative methods as the primary method However, the two types are often given equal weight when evaluating fidelity and assessing implementation barriers and facilitators 16

17 Different methods may be used to answer the same question ( convergence ) or related questions ( complementarity ) Convergence may occur in one of two forms: Triangulation Transformation 17

18 Creswell (2012) has identified several mixed method design typologies Five are particularly relevant to D/I research: Convergent Parallel Design Explanatory Sequential Design Exploratory Sequential Design Embedded Design Multiphase Design 18

19 Simultaneous collection of quantitative and qualitative data, then merging the two types together Example: Pragmatic trial to improve the delivery of evidencebased care for the prevention and management of cardiovascular disease in primary care practices using practice facilitation (Liddy et al., 2011) 19

20 Two sequential phases: Quantitative data collected first Followed by qualitative data collection to help explain the quantitative data Example: Study to evaluate a workshop designed to increase practitioner awareness of best practice recommendations for secondary stroke prevention and identify barriers to implementation in rural practices (Warner et al., 2010) 20

21 Two sequential phases, but in opposite order: Qualitative data collected first Followed by quantitative data collection to provide breadth for emergent relationships Example: Study examining clinician s perceptions about delivering preventive counseling in a brief primary care encounter (Sussman et al., 2006) 21

22 Data collection may be either concurrent or sequential One form of data is embedded within another form, and thus one form is supportive of the other Example: Study to better understand the lack of compliance with mammography screening and design intervention strategies (Puschel and Thompson 2011) 22

23 A series of phases or separate studies, each of which may use a combination of sequential and/or concurrent phases Example: Development and evaluation of an education program for health care professionals and community leaders on how to design, implement, and evaluate a fall prevention program (Scott et al. 2011) Recent increase in innovative implementation research using multiphase designs 23

24 Why are mixed methods not more commonly implemented? Difficulty of gathering and/or accessing data that are nonlinear, or time- or interactionintensive (e.g., process data) Siloization of methodological expertise 24

25 Barriers to publishing or otherwise disseminating results of mixed methods studies Writing stage: Word limits constrain discussion Review stage: Appropriate reviewers difficult to find Review stage: Quality difficult to assess 25

26 Also limitations to how effectively results will be disseminated Results often segregated in different publications that reach limited and often nonclinical audiences Stange et al. (2006) suggest five ways to better integrate the publication of mixed method research 26

27 Improve connection between separate articles Publish separate articles in same journal Publish an integrated single article Publish three articles Develop online discussion 27

28 Pragmatic trials and D/I research key to improving health and healthcare delivery Focus on real-world applications of interventions shown to be efficacious under artificial conditions Understanding of the context in implementation is necessary Quantitative data should not stand alone without an understanding of the impact of context on the results Qualitative data alone are likely to provide limited insight Combining strengths enables translation to real-world settings 28

29 Contact me at: 29

30 Creswell, J. W Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research. Boston, MA: Pearson Education Inc. Creswell, J. W., M. D. Fetters, and N. V. Ivankova "Designing a mixed methods study in primary care." Ann Fam.Med 2(1): Glasgow, R. E., C. Vinson, D. Chambers, M. J. Khoury, R. M. Kaplan, and C. Hunter "National Institutes of Health Approaches to Dissemination and Implementation Science: Current and Future Directions." Am J Public Health 102(7): Liddy, C., W. Hogg, G. Russell, G. Wells, C. D. Armstrong, A. Akbari, S. Dahrouge, M. Taljaard, L. Mayo-Bruinsma, J. Singh, and A. Cornett "Improved delivery of cardiovascular care (IDOCC) through outreach facilitation: study protocol and implementation details of a cluster randomized controlled trial in primary care." Implement.Sci 6: 110. Puschel, K. and B. Thompson "Mammogram screening in Chile: using mixed methods to implement health policy planning at the primary care level." Breast 20 Suppl 2: S40-S

31 Rubenstein, L. V. and J. Pugh "Strategies for promoting organizational and practice change by advancing implementation research." J Gen.Intern Med 21 Suppl 2: S58-S64 Scott, V., E. Gallagher, A. Higginson, S. Metcalfe, and F. Rajabali "Evaluation of an evidence-based education program for health professionals: the Canadian Falls Prevention Curriculum(c) (CFPC)." J Safety Res 42(6): Stange, K. C., B. F. Crabtree, and W. L. Miller "Publishing multimethod research." Ann Fam.Med 4(4): Sussman, A. L., R. L. Williams, R. Leverence, P. W. Gloyd, Jr., and B. F. Crabtree "The art and complexity of primary care clinicians' preventive counseling decisions: obesity as a case study." Ann Fam.Med 4(4): Warner, G., J. Harrold, M. Allen, and R. Lyons "Secondary stroke prevention best practice recommendations: exploring barriers for rural family physicians." Can J Rural.Med 15(4): Wisdom, J. P., M. A. Cavaleri, A. J. Onwuegbuzie, and C. A. Green "Methodological reporting in qualitative, quantitative, and mixed methods health services research articles." Health Serv.Res 47(2):