Fatigue as a Symptom of Coronary Heart Disease. ANN L. ECKHARDT B.S.N., Illinois Wesleyan University, 2003 THESIS

Transcription

1 Fatigue as a Symptom of Coronary Heart Disease BY ANN L. ECKHARDT B.S.N., Illinois Wesleyan University, 2003 THESIS Submitted as partial fulfillment of the requirements for the degree of Doctor of Philosophy in Nursing Sciences in the Graduate College of the University of Illinois at Chicago, 2013 Chicago, Illinois Defense Committee: Julie Zerwic, Chair and Advisor Holli DeVon Jennifer Greene, University of Illinois at Urbana-Champaign Mariann Piano Catherine Ryan

2 I wish to dedicate this dissertation to my family. First, to my loving and supportive husband who did not always understand the stress of doctoral study, but was always there when I needed him. Also to my children who, luckily, are too young to remember this time in their mother s life. They helped provide me with a balance and remembrance that there are more important things than work. Finally, I dedicate this dissertation to my grandparents whose love and support I cherish. Without their guidance throughout my life, I would not be the person I am today. ii

3 ACKNOWLEDGEMENTS I would like to thank and acknowledge those who supported me throughout my doctoral studies at the University of Illinois at Chicago College of Nursing. I am grateful to my committee chair, Dr. Julie Zerwic, who gave of her time, knowledge, and guidance as she supported me through this endeavor. I also give thanks to my committee members, Drs. Mariann Piano, Cathy Ryan, Holli DeVon, and Jennifer Greene, who provided valuable feedback and guidance as I strove to become a better researcher and scientific writer. I would like to acknowledge Ann Ross and Anne Burns who were research assistants and helped tremendously during the data collection process. Finally, I would like to thank both the University of Illinois at Chicago Medical Center in Chicago, IL and Carle Foundation Hospital in Urbana, IL for their support during data collection. This research was completed with support from the Midwest Nursing Research Society and Sigma Theta Tau International. Without grant funding, this research would not have been completed as initially conceptualized. ALE iii

4 TABLE OF CONTENTS MANUSCRIPT PAGE I. Introduction A. Mixed methods purposes and design typologies... 1 B. Quality evaluation in mixed methods research... 1 C. Describing fatigue in coronary heart disease: A mixed methods approach 2 D. Fatigue and predictors of fatigue in coronary heart disease... 3 II. III. Purposes and Design Typologies in Mixed Methods A. Purposes for Mixing Methods Triangulation Complementarity Development Initiation Expansion... 9 B. Mixed Methods Design Typologies C. Conclusion D. Cited Literature Quality Evaluation in Mixed Methods Research A. Background B. O Cathain s Conceptual Framework Domain 1: Planning Quality Domain 2: Design Quality Domain 3: Data Quality Domain 4: Interpretive Rigor Domain 5: Inference Transferability Domain 6: Reporting Quality Domain 7: Synthesizability Domain 8: Utility C. Case Study Exemplar study Application of the 8 Domains a. Planning quality b. Design quality c. Data quality d. Interpretive rigor e. Inference transferability f. Reporting quality g. Synthesizability h. Utility D. Conclusions E. Cited Literature iv

5 TABLE OF CONTENTS (continued) IV. Describing Fatigue in Coronary Heart Disease: A Mixed Methods Approach A. Fatigue and Coronary Heart Disease B. Conceptual Framework C. Research Design D. Methods Design, Sample, and Setting Quantitative Measurement a. Fatigue b. Mood c. Health Related Quality of Life Quantitative Analysis Qualitative Data Collection Qualitative Analysis Mixed Methods Analysis E. Results Demographic Characteristics Timing, Distress, Intensity, and Quality of Fatigue a. Timing b. Distress c. Intensity d. Quality Quality of Life and Fatigue F. Integrated Analysis G. Discussion H. Strengths I. Limitations J. Conclusion K. Cited Literature V. Predictors of Fatigue in Coronary Heart Disease A. Fatigue and Coronary Heart Disease B. Methods Sample, Setting, and Procedures Instruments a. Demographic data questionnaire b. Medical record review form c. Fatigue Symptom Inventory d. Patient Health Questionnaire e. Medical Outcomes Study Short Form Analysis C. Results Demographic characteristics Fatigue in Patients with Coronary Heart Disease a. Interference from fatigue v

6 TABLE OF CONTENTS (continued) b. Fatigue intensity Fatigue and Health Related Quality of Life D. Discussion E. Strengths and Limitations F. Conclusion G. Cited Literature APPENDICIES Appendix A Appendix B VITA vi

7 LIST OF TABLES TABLE PAGE I. PURPOSES OF MIXED METHODS RESEARCH. 5 II. III. IV. INTEGRATED DESIGN TYPOLOGY FOR MIXED METHODS FROM LEECH AND ONWUEGBUZIE(2009) QUALITY DOMAINS FOR ASSESSMENT OF MIXED METHODS STUDIES. 23 DEMOGRAPHIC AND CLINICAL CHARACTERISTICS OF THE SAMPLE.. 45 V. FATIGUE INTENSITY AND INTERFERENCE SCALE ITEMS VI. VII. VIII. FATIGUE SYMPTOM INVENTORY INTERFERENCE SCALE ITEMS AND PHQ-9 SCORE.. 48 FATIGUE SYMPTOM INVENTORY COMPOSITE AND SF-36 VITALITY ITEMS.. 50 COMPARISON OF FATIGUE SYMPTOM INVENTORY COMPOSITE AND QUALITY OF LIFE SCORES.. 52 IX. INTEGRATED DATA ANALYSIS 55 X. DEMOGRAPHIC AND CLINICAL CHARACTERISTICS. 72 XI. PREDICTORS OF FATIGUE. 73 XII. FATIGUE INTENSITY AND SF-36 SUBSCALES.. 75 XIII. SHORT FORM 36 COMPARISON OF CHD PATIENTS TO NORMS BY AGE FOR WOMEN AND MEN. 76 XIV. SHORT FORM 36 COMPARISONS WITH NORMS FOR HYPERTENSION AND RECENT MYOCARDIAL INFARCTION. 77 vii

8 LIST OF ABBREVIATIONS ACS CHD CI FSI HR IHD M MI PHQ-9 PI QUAN QUAL RA SD Acute coronary syndrome Coronary heart disease Confidence interval Fatigue Symptom Inventory Hazard ratio Ischemic heart disease Mean Myocardial infarction Patient Health Questionnaire-9 Principal investigator Quantitative methodology Qualitative methodology Research assistant Standard deviation SF-36 Medical Outcomes Study Short Form 36 SPSS Statistical Package for Social Sciences viii

9 SUMMARY It has been suggested that fatigue is a prodromal symptom of myocardial infarction and a potential indicator of coronary heart disease (CHD). However, no published research has examined fatigue in the stable CHD population. The purpose of this mixed-methods study was to describe fatigue reported by CHD patients using the theory of unpleasant symptoms as a conceptual framework. The dimensions in the theory of unpleasant symptoms guided data collection decisions and each theory dimension was represented by one or more types of data that were collected. All participants (N = 102) completed the Fatigue Symptom Inventory, Medical Outcomes Study Short Form 36 (SF-36), and Patient Health Questionnaire-9 (PHQ-9) to measure fatigue, health-related quality of life, and depressive symptoms, respectively. Each participant also completed a demographic data questionnaire and the researcher completed a medical record review. Semi-structured interviews were completed with a subset of the sample in order to add descriptive depth. Using the Fatigue Symptom Inventory Interference Scale and cut off points determined by a previous study of fatigue and myocardial infarction, participants were sampled from low (n= 5), moderate (n= 4), and high (n=4) fatigue groups for qualitative interviews. Integrated qualitative and quantitative findings confirmed that stable CHD patients experience high levels of fatigue. Seventy-two percent of the sample reported fatigue interfering with their general activity level. Over 40% of participants reported fatigue more than half the days each week with fatigue lasting more than half the day. Participants who reported a PHQ-9 score > 4, indicating depressed mood, reported higher levels of fatigue (χ 2 = 20.45; p <.0001). The mean fatigue score in the sample was indicative of clinically meaningful fatigue. Those participants with higher levels of fatigue also reported lower quality of life on all 8 subscales of the SF36. Interestingly, participants who reported no fatigue on standard measures or during the ix

10 interview reported needing more frequent breaks and taking naps indicating that participants did not always link frequent breaks and naps with feelings of fatigue. This is the first study of stable CHD and fatigue. Interestingly, when stable CHD patients were compared to published data for cancer patients undergoing active treatment (a population known to experience significant fatigue), they reported similar levels of fatigue. Similar to published literature, fatigue was associated with depression and lower quality of life. Women reported higher levels of fatigue than men, but, after controlling for depression, these differences were not significant. Descriptive depth was added to this study by combining qualitative and quantitative methods. Participants did not always report fatigue on standardized measures, but reported fatigue during interviews. This finding illuminates the need for clinicians to ask not only whether a person is experiencing fatigue, but if they have compensated for fatigue in some way. Relying on standardized measures of fatigue provides an incomplete picture of fatigue in this population. More research is needed to further explore the effect of depression on gender differences and determine if other confounding variables like sleep quality and functional status have an effect on fatigue. x

11 I. Introduction This dissertation consists of four manuscripts that will be submitted for publication. The first two manuscripts are methodology manuscripts focusing on various constructs within mixed methods research. The third manuscript is meant to provide a thorough description of fatigue in the presence of coronary heart disease (CHD) using a mixed methods approach, and the final manuscript is a discussion of predictors of fatigue in patients with CHD. Presented below are the abstracts for each manuscript. A. Mixed methods purposes and design typologies The purpose of this methodology paper is to provide a discussion of purposes and design typologies for mixed methods research. Five purposes for mixing methods are provided along with an example from the literature of each purpose. A discussion of the various typologies available for mixing methods follows with the focus being on the most recent and comprehensive typologies. A talbe presents each typology, the purpose it can be used with, and an example from the literature. For mixed methods to be seen as more than a technical approach, it is imperative that researchers using mixed methods thoroughly describe their methods and reasons for mixing. This paper presents only one purpose and typology, but both are the most thorough that have been established to date. B. Quality evaluation in mixed methods research Evaluation of the quality and rigor of mixed methods is a topic of debate. No standard approach to the evaluation of quality in mixed methods exists. The purpose of this paper is to provide an overview of common evaluation standards for mixed methods and then discuss the most recently proposed framework in depth. O Cathain s framework, based on inference quality, 1

12 2 defines 8 domains to assess the quality of mixed methods manuscripts. A discussion of each of the domains is followed by a case study in which the domains are applied to a published article in the literature. While the framework provides an outline for quality assessment from planning through publication, it is cumbersome and impractical. C. Describing fatigue in coronary heart disease: A mixed methods approach Fatigue is a disabling symptom of a variety of acute and chronic conditions. While fatigue has been identified as a symptom of acute cardiac events, it has not been studied as a symptom of stable coronary heart disease (CHD). The purpose of this student was to describe fatigue in patients with stable CHD using the Theory of Unpleasant Symptoms as the conceptual framework. A secondary purpose was to determine whether fatigue had an impact on quality of life. We used a mixed methods descriptive design for the purpose of complementarity. The design typology was a partially mixed sequential dominant status design (QUAN qual). No consistent pattern of daily fatigue was discovered; however, over 40% of the sample reported fatigue more than half of the days of the week, lasting more than half the day. Depressed mood was significantly correlated with fatigue intensity and interference from fatigue. Qualitative interviews provided descriptive depth. Interestingly, taking more frequent breaks was not identified by participants as a consequence of fatigue. Similar to the literature, depressed mood and female gender were correlated with fatigue. Stable CHD patients in this study reported fatigue levels similar to those patients undergoing active cancer treatment. Measurement of fatigue is challenging due to the subjective nature of the symptom. The combination of qualitative and quantitative methods provided a more complete depiction, but more research is needed to determine the extent to which fatigue impacts the stable CHD population.

13 3 D. Fatigue and Predictors of Fatigue in coronary heart disease Fatigue has been identified as a potential predictor of cardiac events, but has not been studied as a symptom of stable coronary heart disease (CHD). This descriptive study aimed to determine predictors of fatigue in stable CHD, identify gender differences, and examine the link between quality of life and fatigue. We used a cross sectional descriptive design. Participants (N = 102) completed the Fatigue Symptom Inventory, Patient Health Questionnaire-9, and Medical Outcomes Study Short Form 36. Age, gender, depressed mood, marital status, and smoking history were correlated with fatigue. Fatigue had a negative impact on quality of life in this population. Consistent with previous research, participants who reported higher levels of depressed mood or were female reported higher levels of fatigue. Interestingly, no gender differences were apparent after controlling for depression. More research is needed to understand fatigue in stable CHD and the impact depressed mood has on fatigue in this population.

14 II. Manuscript #1: Purposes and Design Typologies in Mixed Methods A. Purposes for Mixing Methods Mixed methods research is gaining popularity as health sciences researchers study increasingly complex phenomena. While mixed methods may be useful in the study of complex phenomena, design choices should be carefully considered and mixed methods should only be used when a single method is not suitable to answer the research question (Klassen, Creswell, Plano Clark, Smith, & Meissner, 2012). There are many variations of mixed methods research, and a familiarity with the purposes and design typologies is important as researchers select the best method to answer the proposed question. The definition of mixed methods varies depending on the point at which the mixing occurs, the primary paradigm used, and the design choice. For this paper, mixed methods research is defined as a research methodology that uses two or more means or procedures (e.g., standardized questionnaires and in-depth interviews) and employs rigorous quantitative and qualitative methods (Creswell, Klassen, Plano Clark, & Smith, 2011). This paper will explore the 5 purposes for mixing methods using examples from current literature to demonstrate each method and a short discussion of the most recently recommended design typology will follow. Researchers who conducted a state of the science review of mixed methods identified five purposes for mixing methods (Greene, Caracelli,& Graham,1989 Greene,2007) including: triangulation, complementarity, development, initiation, and expansion. Each of the purposes for mixing has a distinct rationale for its use and, depending on the research question proposed, each can be used in a wide variety of studies (Greene, 2007; Greene, Caracelli, & Graham, 1989)[see TABLE I. PURPOSES OF MIXED METHODS RESEARCH]. 4

15 5 TABLE I. PURPOSES OF MIXED METHODS RESEARCH Purpose Description Triangulation Seek convergence, corroboration Use methods with offsetting biases to increase validity Different methods measure the same phenomenon Methods must be done concurrently Complementarity Seek deeper understanding by using mixed methods to assess different facets of a complex phenomenon Use methods that tap different dimensions of the same phenomenon Development Results from one method are used to inform the other method Methods are implemented sequentially Methods assess the same set of phenomena, but the second method may not assess all of the phenomena assessed by the first Useful in instrument development and sample selection Initiation Respects divergence Both methods assess the same phenomenon, but the results are discordant Seeking potential divergence as a way to more fully understand a complex phenomenon Expansion Different methods are used to assess different phenomena Methods are selected for their congruence with the study foci 1. Triangulation Triangulation seeks convergence and corroboration between different methods. Triangulation can be used to offset biases and shortcomings of one method by combining it with another method. If study results using triangulation converge, then the researcher can have increased confidence in the findings of the study (Greene et al., 1989; Greene, 2007). For example, Riegel, Dickson, Kuhn, Page, and Worrall-Carter (2010a) completed a cross-sectional, comparative mixed methods study of barriers and facilitators to heart failure self-care. 27 adults with heart failure completed survey instruments to measure self-care maintenance, management and confidence, mood, and social support. After completion of quantitative instruments, subjects

16 6 completed in-depth interviews. Data were analyzed using appropriate statistical procedures for quantitative data and narrative analysis was used to analyze the qualitative data. After separate quantitative and qualitative analyses were completed, data were integrated and analyzed for concordance by comparing qualitative themes with quantitative results. Concordance was confirmed between qualitative and quantitative findings thus enhancing confidence in the results. Throughout the results section, researchers provided integrated analyses supporting the successful use of triangulation in the study. For example, men reported more confidence in self-care abilities during the interviews including the ability to assess edema and titrate diuretics appropriately. Women reported less confidence in their assessment ability and would not initiate treatment without consultation. This finding was confirmed in the quantitative analysis in which men reported significantly higher self-care confidence than women. Along with reporting concordant findings, Riegel et al. (2010a) reported the percent congruence or agreement between findings for each construct. 2. Complementarity Complementarity is used to elaborate and clarify complex phenomenon. Mixed methods are used to gain information about different facets of the same phenomenon (Greene, 2007). As an example, Lorvick et al. (2012) described the use of complementarity in their mixed methods research of sexual pleasure and sexual risk taking among women using methamphetamine. Qualitative and quantitative methods were thoughtfully mixed to ensure each method complemented the other. Subjects for the quantitative portion of the study were recruited using respondent-driven sampling with a group of initial subjects helping to recruit other women who met inclusion criteria. Using purposive and strategically targeted sampling, women were recruited from the quantitative group for the in-depth interview. Quantitative data included

17 7 prevalence of different sexual behaviors, patterns of drug use, and perceptions of the impact methamphetamine had on sexual behaviors. Qualitative data, gathered through the interview, included contexts and circumstances of unprotected sex. Integrated qualitative and quantitative results added depth and richness to findings by elaborating on quantitative findings. For example, the Subjective Experience of Meth Sex Scale enabled women to answer, on a Likert scale, whether using methamphetamine influenced sexual thoughts and behavior. Subjects who were interviewed were asked about sexual behaviors and thoughts and the answers to the questions provided a deeper understanding of quantitative findings. Quotes from the interviews were used to illustrate quantitative findings. Researchers in this study did not seek convergence, but instead a deeper understanding of the phenomena of interest through the use of qualitative and quantitative methods that complemented one another. Qualitative and quantitative results were presented separately in the results section and integrated analyses were presented during the discussion. 3. Development Another mixed methods purpose is development, in which the results from one method are used in the development of the other method. Each method is employed sequentially to capitalize on its strengths (Greene et al., 1989; Greene, 2007). As an example, Peterson et al. (2012) employed development in their study to design and test interventions in three chronic disease populations. This three phase study included qualitative data collection, pilot testing of the developed intervention, and finally a randomized control trial. During phase 1 (qualitative), the goal was to elucidate facilitators and barriers for behavior change. Researchers also aimed to identify important values, attitudes, and beliefs about living with a chronic disease. Responses from this phase of the study were then used to tailor study methods and interventions for phases

18 8 2 and 3. Phase 2 of the study was a pilot with the aim of testing specific components of the intervention that had been tailored based on qualitative results. During this phase, both qualitative and quantitative data were collected to ensure the intervention was effective. The final phase of this research involved a randomized controlled trial. Subjects were randomly assigned to either the control group, receiving information only, or the intervention group. During the final phase of the study, only quantitative data were collected, as researchers aimed to determine whether the intervention was successful. In this research, development proved to be useful in identifying, constructing, and testing successful interventions. 4. Initiation Initiation is the use of multiple methods to explore different areas of the same complex phenomenon, but the overall result is to identify dissonance or divergence (Greene, 2007). Initiation studies are not necessarily planned, but divergent qualitative and quantitative results may occur in studies where researchers had a different initial purpose in mind. The divergent results can be generative as they lead the researcher to new questions. As an example, Wagner, Davidson, Pollini, Strathdee, Washburn, and Palinkas (2012) reported a discrepancy in their qualitative and quantitative findings in a study of intravenous drug users. In this study, the researchers chose a sequential mixed methods design using exploratory qualitative data to inform the development of items for a quantitative questionnaire. Given the initial aim of the study, the mixed methods purpose would likely have been termed development, as the researchers goal was to use the initial qualitative interview to develop quantitative questions. However, in the qualitative portion of the study, nearly all participants reported similar risky injection events where they were trying to avoid withdrawal symptoms by using unclean injection equipment. In the quantitative analysis, the risk of withdrawal was an inconsistent reason for risky injection

19 9 behavior. Instead of allowing the discrepancy between findings to discredit either method, the researchers spent more time in integrated analysis to determine potential reasons for the inconsistency. The inclusion of qualitative and quantitative methods led to divergent findings regarding risky injection behavior which informed future directions and changed the potential intervention. The item related to withdrawal as a reason for using unclean injection equipment was removed from the scale and further reliability and validity testing was undertaken. The researcher also spent time examining the role of his 2 year relationship with many of the clients as a potential confounder during the interviews (Wagner et al., 2012). While initiation was not the intended purpose of the mixed methods study, the dissonance between results (initiation) led to generative conclusions. 5. Expansion In studies that employ expansion, different methods are used to look at different components of a study, thus increasing the overall scope (Greene et al., 1989; Greene, 2007). In an expansion design, researchers would choose the most appropriate method to measure the phenomena included in the study. While not prevalent in health research, this design allows researchers to broaden the scope of the overall study, while choosing the most appropriate method to measure each phenomenon, thus enhancing understanding. Finding expansion designs in health sciences research is a difficult task as most researchers focus on a single phenomenon that is measured in multiple ways. Kligler et al. (2011) provides the best example of expansion. In a study of the impact of the Urban Zen Initiative on patient experiences, Kligler et al. (2011) reported using a quasiexperimental design to determine whether an intervention aimed at improving the healing

20 10 environment could improve outcomes. The researchers, however, included a qualitative component making this a mixed methods study. The quantitative component of this study focused on the measurement of mood and quality of life, while the qualitative interviews focused on the specific experience of hospitalization and the alternative therapies offered by the Urban Zen Initiative. While the methods provide complementary information, they did not measure the same phenomenon or constructs of interest thus expanding the overall scope of the study. Unfortunately, information from the qualitative and quantitative components are separated in the results section and no integrated analysis is discussed. The use of an integrated analysis would strengthen this study and perhaps provide more useful information as the researchers continue work on the Urban Zen Initiative. B. Mixed Methods Design Typologies Various scholars have suggested typologies for designing mixed methods research studies (Cook, 1985; Creswell, 2003; Greene, 2007; Morse, 2003), but each typology shares three major dimensions: 1) the point at which information is mixed, 2) parity of the methods used, and, 3) the sequence in which the methods are employed (Greene, 2007). Having an understanding of design typologies available will help researchers plan a study that can best answer the research question and help consumers understand how the study was conducted. Most recently, Leech and Onwuegbuzie (2009) published an integrated typology which was based on a review of published literature regarding mixed methods design typologies. They proposed a three dimension typology which focuses on variations in timing (concurrent or sequential), level of mixing (partially or fully mixed), and emphasis of approaches (equal status versus dominant status). The proposed dimensions map to many of the previously recommended design typologies making it easy to incorporate previously used typologies into their

21 11 recommendation. The first dimension of this typology refers to when the data are collected. Mixed methods researchers must decide whether to collect all data at the same time (concurrent) or at different times (sequential). The second dimension refers to whether the researchers are mixing throughout the research (fully mixed) or whether mixing occurs only after final data analysis (partially mixed). The final dimension refers to whether qualitative and quantitative components of the research are equal in status or if the research has chosen to have a dominant method. Using this three dimensional typology, a total of 8 design types are noted partially mixed concurrent equal status, partially mixed concurrent dominant status, partially mixed sequential equal status, partially mixed sequential equal status, fully mixed concurrent equal status, fully mixed concurrent dominant status, fully mixed sequential equal status, and fully mixed sequential dominant status. Design decisions are made throughout the planning of research and this typology provides an algorithm for decision-making. This algorithm provides guidelines for decision making and the researcher cannot move on to the next step in planning until the previous step has been determined. TABLE II notes a listing of each design type and an example from the literature. Examples included vary from health sciences to psychology to teaching literature. While many studies within the health sciences are using mixed methods, very few use fully mixed designs, instead opting to keep methods separate until analysis has been completed.

22 12 TABLE II. INTEGRATED DESIGN TYPOLOGY FOR MIXED METHODS FROM LEECH AND ONWUEGBUZIE (2009) Design Example Description Most Appropriate Example Notation Purpose(s) P1. Partially mixed concurrent equal status design QUAN + QUAL P2. Partially mixed concurrent dominant status design P3. Partially mixed sequential equal status design Quan + QUAL QUAN QUAL Simultaneous data collection Approximately equal weight to qualitative and quantitative components Mixing occurs after analysis of each data type Simultaneous qualitative and quantitative components Either qualitative or quantitative is dominant Mixing occurs after analysis of each method separately Data collection from one method occurs prior to the other Approximately equal weight to qualitative and quantitative components Mixing occurs after analysis of each method separately Triangulation Complementarity Initiation Expansion Complementarity Initiation Expansion Development Complementarity Expansion Initiation QUAN + QUAL: Riegel et al. (2010b) Quantitative and qualitative data collection occurred almost simultaneously. Integration of methods did not occur until analysis of each method was complete. Methods were of equal status. Quan + QUAL: Riegel et al. (2010a) Quantitative and qualitative collection occurred almost simultaneously. Integration of methods did not occur until analysis of each method was complete. Qualitative was the dominant method. QUAN QUAL: Lim, Biak, & Ashing-Giwa (2012) Quantitative data collection occurred prior to qualitative. Approximately equal weights were given to the methods. Mixing occurred after analysis of each method individually.

23 13 TABLE II (continued) P4. Partially mixed sequential dominant status design F1. Fully mixed concurrent equal status design F2. Fully mixed concurrent dominant status design Qual QUAN QUAN + QUAL Quan + QUAL Data collection from one method occurs prior to the other Either qualitative or quantitative is dominant Mixing occurs after analysis of each method separately Qualitative and quantitative methods are mixed within 1 or more of the following: research objective, type of data and operations, type of analysis, and type of inference Qualitative and quantitative phases are mixed concurrently at one or more stages Both elements have equal weight Qualitative and quantitative methods are mixed within 1 or more stages Qualitative and quantitative phases are mixed concurrently Either qualitative or quantitative is dominant Development Complementarity Expansion Initiation Triangulation Complementarity Initiation Expansion Complementarity Initiation Expansion QUAN qual: Kligler et al. (2011) Quantitative data collection prior to qualitative data collection. No mixing until analysis phase. Quantitative component was dominant. QUAN + QUAL: Daley & Onwuegbuzie (2004) Qualitative and quantitative data collected simultaneously using the same instrument. Methods were mixed at the objective, data analysis, and inference stages. Used what they called six-stage concurrent mixed methods analysis to integrate throughout the analysis process. Methods were considered equal. QUAN + Qual: Collins (2000) Qualitative and quantitative data collected simultaneously using an instrument specifically developed for the study. Methods were mixed at the objective, data analysis, and inference stages. Quantitative was dominant.

24 14 TABLE II (continued) F3. Fully mixed sequential equal status design F4. Fully mixed sequential dominant status design QUAN QUAL Qual QUAN Qualitative and quantitative methods are mixed within 1 or more stages Qualitative and quantitative phases occur sequentially Both elements have equal weight Qualitative and quantitative methods are mixed within 1 or more stages Qualitative and quantitative phases occur sequentially Either qualitative or quantitative is dominant Development Complementarity Expansion Initiation Development Complementarity Expansion Initiation QUAN qual: Taylor & Tashakkori (1997) Quantitative data collected first to group people for the qualitative data. Methods mixed throughout the study with equal qualitative and quantitative components. QUAL quan QUAL: Waysman & Savaya (1997) Data were collected sequentially with each method informing the next phase of the study. Qualitative components carried more weight than the quantitative component.

25 15 Leech and Onwuegbuzie (2009) recommend the use of a notation system for mixed methods that was first credited to Morse (1991). While various scholars have proposed notation systems for mixed methods research, they all represent modifications to the original system developed by Morse. This system recommends the use of qual for qualitative research and quan for quantitative research. If a method is in all capital letters, it is the dominant method. A plus sign is used for concurrent data collection and an arrow for sequential data collection. This notation system provides 24 different combinations for mixed methods design and can be easily incorporated into the typology. Using this system to note priority and relationship between components of a study help consumers of the research to understand how the study was designed. C. Conclusion While determination of the research question is always the first step to study design, it is imperative that researchers are thoughtful about the combination of methods. Incorporating the purpose for mixing and a design typology in manuscripts is an important step to ensure that mixed methods is seen as a distinct research methodology, not simply a technique. Qualitative and quantitative manuscripts require authors to state their design (two-armed quasi-experiment or ethnography) and this statement provides a great deal of information about design choices and the likely goal of the study. For mixed methods research to gain the same recognition as qualitative and quantitative methods, it is important that researchers state their mixed methods purpose and design typology within the published manuscript. Only time will tell whether the creation of a mixed methods language is successful, but an important first step is for authors of mixed methods manuscripts to begin using the language of the discipline.

26 16 Researchers do not always label the mixed methods being used in a study; however, knowing what the initial purpose was can be helpful to the consumers of research. Understanding the purpose of the method can help consumers understand the method choices, data collection decisions, and overall study design. Certain mixed methods require data to be collected concurrently (triangulation) while other purposes do not delineate rules for data collection timing (complementarity). It is imperative that mixed methods researchers report all aspects of the mixed methods choices to ensure that mixed methods is realized as a distinct research methodology and not simply a technical approach.

27 17 D. Cited Literature Collins, K.M.T. (2000). Implementing mathematics curricula standards: Effective instruction for all students? Paper presented at annual meeting of the mid-western Educational Research Association, Chicago, IL. Cook, T. D. (1985). Postpositivist critical multiplism. In R. L. Shotland, & M. M. Mark (Eds.), Social science and social policy (pp ). Thousand Oaks, CA: Sage Publications. Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods (2nd ed.). Thousand Oaks, CA: Sage Publications. Creswell, J.W., Klassen, A.C., Plano Clark, V.L., & Smith, K.C. for the Office of Behavioral and Social Sciences Research. (2011). Best Practices for Mixed Methods Research in Health Sciences. National Institutes of Health. Retrieved October 1, 2012 from Daley, C.E. & Onwuegbuzie, A.J. (2004). Attributions toward violence of male juvenile delinquents: A concurrent mixed methods analysis. Journal of Social Psychology, 144, Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco: Wiley. Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-methods evaluation designs. Educational Evaluation and Policy Analysis, 11(3), Klassen, A.C., Creswell, J., Plano Clark, V.L., Smith, K.C., & Meissner, H.I. (2012). Best practices in mixed methods for quality of life research. Quality of Life Research, 21, doi: /s x

28 18 Kliger, B., Homel, P., Harrison, L.B., Sackett, E., Levenson, H., Kenney, J., Merrell, W. (2011). Impact of the Urban Zen Initiative on patients experiences of admission to an inpatient oncology floor: A mixed-methods analysis. Journal of Alternative and Complementary medicine, 17, doi: /acm Leech, N.L. & Onwuegbuzie, A.J. (2009). A typology of mixed methods research designs. Quality and Quantity, 43, Lim, J., Biak, O.M., & Ashing-Giwa, K.T. (2012). Cultural health beliefs and health behaviors in Asian American breast cancer survivors: A mixed-methods approach. Oncology Nursing Forum, 39, doi: /12.ONF Lorvick, J., Bourgois, P., Wenger, L.D., Arreola, S.G., Lutnick, A., Wechsberg, W.M., & Kral, A.H. (2012). Sexual pleasure and sexual risk among women who use methamphetamine: A mixed methods study. International Journal of Drug Policy, 23, doi: /j.drugpo Morse, J. M. (2003). Principles of mixed methods and multimethod research design. In A. J. Tashakkori, & C. Teddlie (Eds.), The handbook of mixed methods in social and behavioral research (pp ). Thousand Oaks, CA: Sage Publications. Peterson, J.C., Czajkowski, S., Charlson, M.E., Link, A.R., Wells, M.T., Isen, A.M., Jobe, J.B. (2012). Translating basic behavioral and social science research to clinical application: The EVOLVE mixed methods approach. Journal of Counsulting and Clinical Psychology. Advance online publication. doi: /a Riegel, B., Dickson, V. V., Kuhn, L., Page, K., & Worrall-Carter, L. (2010a). Gender-specific barriers and facilitators to heart failure self-care: A mixed methods study. International Journal of Nursing Studies, 47(7), doi: /j.ijnurstu

29 19 Riegel, B., Dickson, V. V., Cameron, J., Johnson, J. C., Bunker, S., Page, K., & Worrall-Carter, L. (2010b). Symptom recognition in elders with heart failure. Journal of Nursing Scholarship: An Official Publication of Sigma Theta Tau International Honor Society of Nursing / Sigma Theta Tau, 42(1), doi: /j x Taylor, D.L. & Tashakkori, A. (1997). Toward an understanding of teachers desire for participation in decision making. Journal of School Leadership, 7, Wagner, K.D., Davidson, P.J., Pollini, R.A., Strathdee, S.A., Washburn, R., & Palinkas, L.A. (2012). Reconciling incongruous qualitative and quantitative findings in mixed methods research: Exemplars from research with drug using populations. International Journal of Drug Policy, 23, doi: /j.drugpo Waysman, M. & Savaya, R. (1997). Mixed methods evaluation: A case study. Evaluation Practice, 18,

30 III. Manuscript #2: Quality Evaluation in Mixed Methods Research A. Background Researchers are tasked with evaluating the quality of the research through the use of wellrespected standards; this can be especially challenging with mixed methods studies. Quantitative researchers use reliability and validity as measures of rigor in their studies (Polit & Beck, 2008); while qualitative researchers use trustworthiness or credibility as a measure of quality (Lincoln & Guba, 1985). The variation in evaluation methods contributes to difficulties in the evaluation of mixed methods research. Researchers are beginning to develop standards to evaluate research using mixed methods, but no single standard has been accepted (O Cathain, 2010).The purpose of this paper is to discuss issues surrounding mixed methods evaluation and, using the most recent framework, apply the criteria to a mixed methods study. Three primary approaches are available for the assessment of quality in mixed methods research; generic, assessment of individual components, and development of a unique set of criteria. The generic approach assesses the quality of the research using one of a number of instruments available. Eleven such instruments exist, but none fully evaluate quality (Katrak, Bialocerkowski, Massy-Westropp, Kumar, & Grimmer, 2004). The second approach, assessing individual components, is most often used in health care research. This approach allows researchers to assess the quality of the quantitative and qualitative components separately. While widely used, this approach fails to consider that mixed methods research is not simply the sum of its parts (Creswell & Plano Clark, 2007). While it is important that each method is rigorous, it is also important that inferences drawn from an integrated analysis are reliable, valid, and 20

31 21 trustworthy. In the final approach, researchers using mixed methods have attempted to create a unique set of quality criteria to address mixed methods research as a whole, instead of as a sum of its parts (O Cathain, 2010). For the past 20 years, scholars in the field of mixed methods research have struggled to create a quality framework that could be applied to assess mixed methods studies. From early publications on the subject, two primary schools of thought have evolved legitimation and inference quality. Legitimation is a conceptual framework to evaluate the quality of mixed methods research. Legitimation includes the quality of the method and inference quality, as described above. It builds on accepted frameworks from qualitative and quantitative traditions. Onwuegbuzie and Johnson (2006) proposed nine different types of legitimation. Each type of legitimation allows for discussion of the complexity inherent in mixed methods designs. Types of legitimation include: sample integration (does the sampling yield quality), inside-outside (the degree to which the researcher is presenting the view of the subject), weakness minimization (how well weaknesses from one approach are compensated for using the second approach), sequential (whether the sequence of methods affects the results), conversion (did qualitizing or quantitizing affect data quality), paradigmatic mixing (were underlying beliefs blended into a usable form), multiple validities (is information gained from each method valid and, when combined, greater than either method individually), and political (is the information gained through the use of both methods valuable to the consumer). Teddlie and Tashakkori (2003) recommended inference quality as way to evaluate mixed methods studies. Inference quality is a term for accuracy which is applied to both inductively and deductively derived conclusions from a study. Through the development of a nomenclature that

32 22 is specific to mixed methods, researchers and consumers have a specific set of guidelines to follow to evaluate the quality of the study (Teddlie & Tashakkori, 2003). Most recently, O Cathain (2010) proposed a quality framework for use in mixed methods research. The core of O Cathain s framework is inference quality (Teddlie & Tashakkori, 2003), identified just as quality. O Cathain (2010) described 8 quality domains that should be used to assess the rigor of a mixed methods study. Each of the 8 domains will be discussed and applied to a study as a demonstration of the way in which the framework evaluates mixed methods quality (see TABLE III QUALITY DOMAINS FOR ASSESSMENT OF MIXED METHODS STUDIES).

33 23 TABLE III. QUALITY DOMAINS FOR ASSESSMENT OF MIXED METHODS STUDIES (O Cathain, 2010) Domain Stage of Study Items Included Definition Planning quality Planning/proposal 1. Foundational element 2. Rationale transparency 3. Planning transparency 4. Feasibility 1. Thorough literature review has guided the research question 2. Rationale for the use of mixed methods research is clear 3. Well thought out plan for all components of mixed methods research 4. Study can be undertaken with available resources (financial, time, Design quality During 1. Design transparency 2. Design suitability 3. Design strength 4. Design rigor Data quality During 1. Data transparency 2. Data rigor 3. Sampling adequacy 4. Analytic adequacy 5. Analytic integration rigor Interpretive rigor Interpretation 1. Interpretive transparency 2. Interpretive consistency 3. Theoretical consistency 4. Interpretive agreement 5. Interpretive distinctness 6. Interpretive efficacy 7. Interpretive bias reduction 8. Interpretive correspondence Inference transferability Interpretation 1. Ecological transferability 2. Population transferability 3. Temporal transferability 4. Theoretical transferability and human) 1. Design should be described using established typology if possible 2. The design must match the research question 3. Design chosen optimizes breadth and depth of study while minimizing overlapping bias between methods 4. Methods being used in a way that is proper for the design 1. Methods are thoroughly described 2. Methods are implemented in a rigorous fashion 3. Sampling technique is described and is appropriate within the context of the method and design choices 4. Analysis techniques are appropriate 5. Integrated analysis is robust 1. Which finding came from which method is clear 2. Inferences consistent with the findings 3. Inferences consistent with current knowledge or theory 4. Based on the findings, others are likely to have the same conclusions 5. Conclusions are more credible than alternatives 6. Inferences from an integrated analysis (qualitative and quantitative) adequately integrate inferences from each method 7. Inconsistencies are discussed 8. Research question and purpose of the study correspond to the conclusions 1. Ability to transfer data to other settings 2. Ability to transfer to another sample or population 3. Ability to use findings in the future 4. Ability to transfer to other ways of measurement

34 24 TABLE III (continued) Reporting quality Dissemination 1. Report availability 2. Reporting transparency 3. Yield 1. Completion of the study within the allocated resources 2. Important aspects of the study reported 3. The use of mixed methods research provided more information than either method could have alone Synthesizability Applicability Scoring system Determination of whether the study provides important enough evidence to be included in systematic reviews Utility Applicability Utility quality The findings are useful to consumers of research or policy makers

35 25 B. O Cathain s Conceptual Framework 1. Domain 1: Planning Quality The first domain for evaluating quality in mixed methods research is planning quality, which addresses how well the study has been planned. Important in this domain is rationale transparency wherein the researchers provide rationale for the use of mixed methods and the intrinsic value of mixed methods for answering the research question. Planning quality also includes planning transparency where key issues and aspects of the design are detailed and the feasibility of the study is determined. In mixed methods, feasibility includes not only issues of time and funds, but also expertise in the various methods proposed (O Cathain, 2010). Because the first domain is apparent at the research proposal stage, it is difficult to assess in published work. Rationale transparency, however, should be apparent in publications. 2. Domain 2: Design Quality Design quality refers to design transparency, design suitability, design strength, and design rigor. Included within this domain are decisions made about the combination of chosen methods. It is important that researchers use different methods to offset bias, minimize weaknesses within each method, and make sure the purpose suits the design choice (O Cathain, 2010). 3. Domain 3: Data Quality Data quality refers to data transparency, data rigor, sampling adequacy, analytic adequacy, and analytic integration rigor. Each area within this domain covers important concepts within data collection and analysis. Methods being used should be described in detail, sampling plans and reasons for sampling decisions should be apparent, and analysis techniques should be appropriate for the type of data being collected. 25