A QUALITY FRAMEWORK FOR CASE STUDY RESEARCH: CONVINCINGNESS

Transcription

1 A QUALITY FRAMEWORK FOR CASE STUDY RESEARCH: CONVINCINGNESS Introduction Case studies are empirical investigations where the area of study is circumscribed to a small number of units, thus enabling the researcher to look in depth at a topic of interest or phenomenon. This style of research generates knowledge that is context-dependent providing a rich and nuanced view of reality (Flyvebjerg, 2006). Adherents (for example, Eisenhardt, 1989; Yin, 2009) extol its value in answering questions that start with how, who and why. Concerns about the quality of case study research, nonetheless, are well entrenched although not necessarily legitimate and which destabilise the contribution that case study research makes to marketing and to management research. The purpose of this paper is to evaluate ways in which research quality is assessed with reference to case study investigation. From this evaluation, an overarching framework that strengthens arguments for the contribution of case study research is proposed. The structure of this paper is as follows: first, an overview of case study research, then a discussion of classical and interpretivist approaches to quality in case study research, concluding with a proposed framework for convincingness as a means of demonstrating overall quality in case study research. Case study research For research to be accepted, published or funded, it must usually meet a number of criteria which may vary according to the epistemology of the audience. Convincing research aims to reach out beyond its methodological foundation to argue its contribution to theory in its field. Case study research consists of empirical investigation using familiar research techniques (Yin, 2009), usually involving the generation of data from more than one source within each case. If the research is investigating multiple cases, the data collection is then repeated in each of the cases. Researchers, therefore, need to demonstrate the rigour of their methods within each set of data, within each case as well as across the cases (Johnston et al. 1999). The researcher is then faced with the challenge of ensuring that each of the data sets meets not only the quality criteria for that particular method but convincing the reader of the overall contribution of the study (Farquhar, 2012). The quality of case study research can be argued from classical and interpretivist perspectives. Classical approaches to research quality Construct validity, internal validity and reliability of research are all tests derived from the natural sciences and which have been considered as means of judging the quality of case study research. Construct validity defines how well a test or experiment measures up to its claims. It refers to whether the operational definition of a variable actually reflects the true theoretical meaning of the concept which it is purporting to measure. In case study research, claims for construct validity can be asserted through the number of data sources that have been assembled. Usual reasons for accessing more than one data source are bias reduction or achieving some degree of external reality, neither of which really address construct validity. Multiple data sources form the basis for triangulation and as such appear more suited to establishing convergent validity (Jick, 1979). A second approach for claiming construct validity is through the establishment of a clear chain of evidence which enables the reader to follow how the researcher went from research question to conclusion (Yin, 2009), bolstered by detail in how the ideas, concepts and relationships that form the basis of the study were addressed (Remenyi et al. 1998). Although these approaches are good research practice, they do not really address the rather precise requirements of construct validity. Internal validity refers to the presence of causal relationships between variables and results and applies at both 1

2 data collection and analysis stages (Gibbert and Ruigrok, 2010). Causal investigation requires that any other explanation of the causal effect is ruled out by testing rival explanations for the findings. Detail about how the researchers triangulated the data, that is, across data types and sources will also be required (Johnston et al. 1999). Eisenhardt (1989) advances the view that case study researchers can argue for internal validity through enfolding literature (p544) where the emerging concepts and theory from case study research are closely examined with the existing literature. Reliability refers to the absence of random error so that if the research was repeated, the researchers would arrive at the same insights. In terms of case study research, transparency and replication are key words in claims for reliability (Gibbert and Ruigrok, 2010). The former can be demonstrated through careful documentation and references to a research database, which Yin (2009) consistently advocates as essential for case study research. Usefully the creation of a database is also thought to be supportive in arguments for construct validity in showing the chain of evidence. Replication questions are addressed through references to the planning and execution of a coherent research strategy in line with accepted protocols. As case study research is concerned with in-depth investigations and is contextually bound, is reliability is a key criterion in assessing its quality? The final quality assessment for case study research under the heading of classical approaches is generalizability, which relates to the belief that theories must be shown to account for phenomena not only in the setting in which they were studied but elsewhere (Remenyi et al. 1998). One of the most frequent criticisms levelled at case study research is that the findings cannot be generalized to a wider population. Swanborn (2010) summarises the problem of generalization both in single and multiple case studies, by querying what can be said about non-researched cases. He questions Yin s (2009) comparison of case study research with experiments, commenting that the variables in case study research are entangled, as opposed to the isolated variables in experiments. He concludes that generalizing from case study research is only tentatively possible but that the problem diminishes if case study research is complementary to more extensive research, which rather undermines the purpose of this choice of research strategy. A critical aspect in the debate is the assertion that multiple cases provide the basis for generalizability (Leonard-Barton, 1990). Johnston et al. (1999) advocate multiple case study research claiming that generalization can be achieved through replication in multiple settings but there is some unhelpful overlap with reliability tests with this view. Other case study researchers (for example, Flyvebjerg, 2006; Siggelkow, 2007) vigorously contest the view that multiple case studies enable any form of statistical generalization. Indeed Yin (2009, p43) is uncompromising in his dismissal of generalizability in case study research as follows, [The] analogy to samples and universes is incorrect when dealing with case studies From this perspective, where he likens case study research to experimental research, Yin (2009) advocates the logic of replication as opposed to the logic of sampling. When a significant finding is made in a case, for this finding to be considered robust, it is necessary to replicate the research in further experiments so that finding can be duplicated. There is some consensus that multiple case studies do not form the same purpose as multiple respondents, instead, the researcher seeks to replicate a particular finding through replicating multiple cases. Generalizability in case study is achieved through the findings being generalized to theoretical propositions, which is known as analytic generalization (Yin, 2009) as opposed to statistical generalization. Analytic generalization is a process where 2

3 generalizing takes place from data to theory rather than from sample data to population. In analytic generalization, the findings are considered in the light of being congruent with or connected to prior theory (Miles and Huberman, 1994). Flyvbjerg (2006) affirms that generalization can be made from single case study research although he considers generalization to be overvalued as a means of developing theory, arguing instead for the force of example in theory development. This section of the paper has appraised the classical approaches for judging the quality of case study research. Some adaptation of these classical approaches has been necessary, revealing some problems with fit, in particular with construct validity. Are these gaps in fit surmountable? Is it appropriate to use these classical measures for case study research? Much case study research leans towards an interpretivist epistemology, the quality procedures for which are now considered within the context of case study research. Interpretivist perspectives Interpretivist researchers take the view that establishing the quality of their study lies not in adopting the criteria favoured by positivist researchers but in espousing principles that are in tune with the epistemological foundations of their work. Unlike classical research, where quality checks follow the standard validity, reliability and generalizability model, interpretivist researchers adopt an eclectic range of quality checks that support the claims for the credibility or trustworthiness of their research. Patton (1989), for example, proposes three elements for addressing credibility: rigorous techniques and methods; researcher credibility and philosophical belief of the researcher. Easterby-Smith et al. (2008) have argued for pluralism in determining quality within qualitative research, whilst expecting to see certain features such as: systematic and demanding processes of enquiry and the creation of distinct value, relative to some theoretical or applied domain. Without entering into the debate of whether qualitative enquiry is interpretivist, these elements nonetheless act as a reminder of a degree of diversity in non-positivist research in terms of quality procedures. Guba and his co-author Lincoln (for example, 1982, 1994) published a stream of work on naturalistic enquiry in which they debate quality procedures qualitative research. Importantly, these criteria can be extended to case study research (Guba and Lincoln, 1982). As a means of demonstrating what they describe as the trustworthiness of qualitative data, they developed four criteria as follows: credibility, transferability, dependability and confirmability. Just as we have considered classical criteria for demonstrating case study research quality, we now do so for these criteria, beginning with credibility. The credibility of research can be demonstrated through the adoption of appropriate and well recognised research methods that form the empirical aspect of the case or cases. Such measures might include evidence of tactics in obtaining data from informants, for example through detailing iterative questioning in data collection recording dialogues rather than merely extracts (Shenton, 2004). Evaluating alternative explanations (Patton, 1999) enhances analysis where the failure to find strong supporting evidence for other accounts helps to increase confidence in the original explanation for the data. In the interpretivist lexicon, the criterion of transferability involves a return to the theme of generalizability. The question of whether the findings of case study research can be transferred to other contexts may not of prime interest to the case study researcher as the strength of case study research rests in the context in which the theory is explored or developed. If the research has been defined or bounded explicitly within the appropriate theory and selection of the case(s) justified as a means of advancing understanding with reference to that theory (Remenyi et al. 1998), then questions of transferability are to some extent answered. Transferability can be further 3

4 covered through extensive background data or thick description (Creswell, 2007) which permits readers to make their own comparisons. According to Lincoln and Guba (1985, p.299) defending the dependability of research comprises seeking the means for taking into account both factors of instability and factors of phenomenal or design induced changes. The researcher thus may make or have to make changes in the way that she collects and analyses data for reasons that were not appreciated at the outset of the research. In research, it is acknowledged that interviewing and observing is an evolving process during which interviewers and observers acquire new insights into the phenomenon of study, which can subsequently influence follow-up questions or narrow the focus for observation (Graneheim and Lundman, 2004). Demonstrating the dependability of research can be achieved through explanation of planning and execution, the relation of operational detail of data gathering not omitting the minutiae of what was done in the field and s reflective appraisal of the project (Shenton, 2004). All of this would be applicable to case study research and is indeed reported as being a key strategy in high quality case study research (Gibbert and Ruigrok, 2010). In arguing for the fourth and final trustworthiness criterion of confirmability, the researcher needs to persuade the reader that the research is not overly influenced by personal values or theoretical inclinations (Bryman, 2001). This is not a plea for moving towards objectivity, which would be an anathema to the interpretivist but instead is about demonstrating that alternative explanations have been considered and the processes described in both credibility and transferability can apply in confirming the research findings. So far, in this paper, the two dominant approaches to assessing the quality of research have been reviewed according to the epistemological assumptions of classical (positivism) and interpretivism. However, case study research is more than a sum of its various data collection modes and indeed these procedures are usually envisaged as applying to mono-method data collection. Although arguments have been elaborated for their application to case study research, it is the contention of this paper that case study research merits an overarching quality approach, where all the data sets are brought together in addressing the research/question. Case study research takes place at two levels, firstly at data collection level, for example a survey or analysis of secondary data. Researchers seek to persuade or even convince but can do this more effectively not at data collection level but at a higher order appraisal of the data. Whilst the individual data sets can be assessed according to the criteria discussed above, how can the strength and hence contribution of the research as a whole be championed across several diverse data sets and cases? Going beyond Researchers seek to make a contribution to their field of study and in so doing try to convince their audience that their research makes a contribution. Case study research can do this most effectively when all recognised quality procedures have been performed on the various data sets and then by arguing for the contribution at a higher level. In order to fulfil these higher level criteria, the idea of convincingness is borrowed from ethnographic research. Its three dimensions of authenticity, plausibility and criticality are harnessed to address the particular requirements of case study research Authenticity Is the reader is being presented with an authentic portrait of what is being studied? Miles and Huberman (1994) draw a parallel between authenticity and internal validity but distinguish between the measurement-oriented tests of validity and, instead, pose such questions as does the study ring true and enable the reader to be there? (p279). Authenticity, later added as a 4

5 fifth criterion to Guba and Lincoln s original criteria, can be demonstrated by the researcher conveying to the reader that she was actually there and genuine to the experience of the research (Golden-Biddle and Locke, 1993). There is an implication in much of the case study literature that the case study researcher has a physical connection to the case(s) such as site visits. This notion of physical connection can be re-envisaged in line with technology waves and so extended to a virtual presence. These connections can be addressed through evidence of context specific information but offers the researcher the opportunity to create a compelling picture. Plausibility Plausibility is concerned with connecting the world of the study with the readers experience, that is does the research make sense to me? In posing this question, the reader wishes to be convinced that the study is dealing with concerns relating to its disciplinary background and, at the same time, making a distinctive contribution to the area (Golden-Biddle and Locke, 1993). Researchers can achieve plausibility in their studies by ensuring that the work relates to readers experiences. The writer should attempt to smooth any likelihood of readers rejecting assertions through providing background debate to controversial assertions in the literature, for example by enfolding the literature (Eisenhardt, 1989). Plausibility is therefore concerned with reducing queries about the research not being in tune with the readers own experience. Criticality At the same time, researchers should be aiming to activate readers to re-examine assumptions (Golden-Biddle and Locke, 1999 p 610). By adopting an in-depth study not only is a researcher seeking to contribute to knowledge in a particular domain but also to unearth through the data collection methods and subsequent triangulation something challenging and profound. Researchers will demonstrate that their work achieves criticality by encouraging readers to re-examine their existing ideas and beliefs, encouraging them to see the world in a new way or through tackling alternative explanations (Siggelkow, 2007)? There is also scope with this criterion for allowing the study to be different things to different people. Researchers can avoid directing readers down a particular path, instead inviting them to decide the meaning of the case. In this way, readers will have to discover their own path and truth inside the case (Flyvebjerg 2006). The contention of this paper is that convincingness as demonstrated through addressing these three criteria provides a higher order quality framework that builds on the lower order quality procedures and enables case study researchers to argue for the contribution of their work in the most persuasive and convincing light. Like any hierarchy of procedures, the higher order procedures can only function if the lower order are in place, therefore concerns about validity, for example, have to be addressed if data collection follows a technique where that test might apply. This paper seeks not only to promote the framework of convincingness but also to argue for quality processes at all levels. This paper has evaluated a range of quality procedures for case study research and owing to the particular requirements of this research strategy proposes an overarching of framework of convincingness to address the research quality issues arising from case study research. The framework blends the range of techniques and data open to the case study researcher with the peculiar strengths of context and depth that the strategy can achieve. 5

6 References Bryman, A. (2001), Social Research Methods, New York, Oxford University Press Inc. Creswell, J. W. (2007), Qualitative Enquiry & Research Design: Choosing Among Five Approaches, Thousand Oaks, Sage Publications. Farquhar, J. D. (2012), Case Study Research for Business, London, Sage Publications. Easterby-Smith, M., Golden-Biddle, K. and Locke, K. (2008), Working with Pluralism: Determining Quality in Qualitative Research, Organizational Research Methods, 11, 3, Eisenhardt, K. (1989), Building Theories from Case Study Research, Academy of Management Review, 14, 4, Flyvbjerg, B. (2006), Five Misunderstandings About Case Study Research, Qualitative Inquiry, 12, 2, Gibbert, M. and Ruigrok, W. (2010), The What and How of Case Study Rigor: Three strategies based on Published Work, Organizational Research Methods, 13, 4, Golden-Biddle, K. and Locke, K (1993), Appealing work: An investigation of how ethnographic texts convince, Organization Science, 4, 4, Graneheim, U.H. and Lundman, B. (2004), Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness, Nurse Education Today, 24, Guba, E. and Lincoln, Y. (1982), Epistemological and Methodological Bases of Naturalistic Enquiry, Education, Technology Research Development, 30, 4, Guba, E. and Lincoln, Y. (1992), Competing paradigms in qualitative research, Handbook of Qualitative Research, Thousand Oaks, Sage Publications. Jick, T. (1979), Mixing Qualitative and Quantitative Methods: Triangulation in Action, Administrative Science Quarterly, 24, 4, Leonard-Barton, D. (1990), A Dual Methodology for Case Studies: Synergistic Use of Longitudinal Single Site with Replicated Multiple Sites, Organizational Change, 1, 3, Lincoln, Y.S. and Guba, E.G. (1985), Naturalistic inquiry, Beverly Hills: Sage. Miles, M. and Huberman, A. (1994), Qualitative Data Analysis (2 nd Edition), Thousand Oaks, Sage Publications. Patton, M. (1999), Enhancing the Quality and Credibility of Qualitative Analysis, Health Services Research, 34, 5, Remenyi, D., Williams, B., Money, A. and Swartz, E. (1998), Doing Research in Business and Management: An Introduction to Process and Method, London, Sage Publications. Shenton, A. (2004), Strategies for ensuring trustworthiness in qualitative data projects, Education for Information, 22, Siggelkow, N. (2007), Persuasion with case studies, Academy of Management Journal, 50, 1, Swanborn, P. (2010), Case Study Research: what, why and how? London, Sage Publications. Yin, R. (2009), Case Study Research: Design and Methods (Fourth Edition), Thousand Oaks, Sage Publications. 6