Introduction to research methods and critical appraisal WEEK 11

Introduction to research methods and critical appraisal WEEK 11

Session objectives Define and understand the term critical appraisal Revisit the 10 questions from week 10 and examine them in more depth Understand some basic research methods terminology Critically appraise an article on either health promotion or health education to be included in your portfolio

What is critical appraisal? In your groups please discuss the following for 10 minutes: 1. What is critical appraisal? 2. When would we use critical appraisal? 3. Why would we use critical appraisal? 4. How do we critically appraise?

The 10 questions

1. Is the study question relevant? Does it address an important topic? Is it relevant to your field of work?

2. Does the study add anything new? Does the study add any new information to the subject? Does this increase the generalisability of previous research? Does this increase the validity of previous research?

Validity and reliability

3. What type of research question is being asked? Does the question include: The group or population of patients? The studied parameter? The outcomes of interest? Is it a question about: The effectiveness of treatment? The frequency of events?

4. Was the study design appropriate for the research question? Effectiveness questions Meta-analyses of well-conducted randomised controlled trials and individual randomised control trials are the most robust Frequency questions Observational studies are the most appropriate study design If the most appropriate or robust design was not followed why might this be?

Qualitative vs. quantitative The aim is a complete, detailed description. The aim is to classify features, count them, and construct statistical models in an attempt to explain what is observed. Researcher may only know roughly in advance what he/she is looking for. Researcher knows clearly in advance what he/she is looking for. Recommended during earlier phases of research projects. The design emerges as the study unfolds. Researcher is the data gathering instrument. Recommended during latter phases of research projects. All aspects of the study are carefully designed before data is collected. Researcher uses tools, such as questionnaires or equipment to collect numerical data. Data is in the form of words, pictures or objects. Data is in the form of numbers and statistics. Subjective - individuals interpretation of events is important,e.g., uses participant observation, in-depth interviews etc. Objective - seeks precise measurement & analysis of target concepts, e.g., uses surveys, questionnaires etc. Qualitative data is more 'rich', time consuming, and less able to be generalized. Quantitative data is more efficient, able to test hypotheses, but may miss contextual detail. Researcher tends to become subjectively immersed in the subject matter. Researcher tends to remain objectively separated from the subject matter.

5. Did the study methods address the most important potential sources of bias? Is there any potential random bias? Is there any potential systematic bias? Were the variables controlled?

6. Was the study performed according to the original protocol? Were the planned number of participants recruited? Did the inclusion or exclusion criteria change? Were there variations in the treatments or interventions? Were there changes to the employed techniques or technologies? Were there changes to the duration of follow up?

7. Does the study test a stated hypothesis? Can the null hypothesis be rejected? This indicates that the results are statistically significant Has all data relevant to the study been reported? Have selected outcomes been omitted? Was the sample size calculated prior to the study to ensure statistically significant results?

8. Were the statistical analyses performed correctly? Is there a methods section? Are the tools used in statistical analysis explained? Is the rationale given? Are any missing results or anomalies clearly explained in the results section?

9. Does the data justify the conclusions? Are the conclusions reasonable when the accumulated data is weighed up? Have sample sizes affected the consideration of statistically significant results? Have the findings been appropriately generalised? Or inappropriately based on the sample subjects? Could there be any misinterpretation around cause and effect?

10. Are there any conflicts of interest? Are the researchers backgrounds made clear? Is there a statement regarding source funding? Are there any potential financial conflicts of interest? Are there any potential personal conflicts of interest? Is there any institutional gain for any of the researchers? If there are potential conflicts of interest, are they likely to have influenced the validity of the study?

Your task You will be given an article on health promotion or health education, please consider the difference between these terms. Using the 10 questions as your structure please also produce an individual piece of work critically appraising your article. Your work should have a short introduction, the 10 questions as headings (with specific considerations for your study under each) and a short conclusion. This a summative piece of work and will be handed in on Friday with your PAD. ANY QUESTIONS?