Welcome to our e-course on research methods for dietitians

Transcription

1 Welcome to our e-course on research methods for dietitians This unit is on Sample size: Also reviews confidence intervals and simple statistics This project has been funded with support from the European Commission. This publication [communication] reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. E-course Sample Size Unit Page 1

2 Authors: Marta Fajó Pascual, University of Zaragoza, Spain Anne de Looy, University of Plymouth, Great Britain This is one unit in a series of several units in research methods. It is intended for students with some previous basic knowledge of biostatistics. E-course Sample Size Unit Page 2

3 Introduction: This unit on Sample size has been written to be used by student dietitians and by teachers for their students. It is not intended to replace student classes but to support revision or provide additional resources to further learning. It is made up of: Written guidance Links to helpful internet materials A research paper for illustration Guided questions and answers to discuss with others Other units are available (see list on and the DIETS2 project would like to have feedback on improvements or additions which could be made to this unit. On our website you will find a place for comments. Thank you, Here we go... 1) In planning or reviewing dietetic research it is important to make sure that the results are credible, believable. In statistical terms, we need the results to be valid (no bias, no systematic error or at least minimized) and reliable (minimized random error). 2) There are many ways to check out if the results are valid. One way is to ask questions about the sampling process or how to select the sample so as to be representative : The difficulty with selecting a sample is ensuring that is representative of the entire target population. Is the sample chosen appropriate to answer the research question? For example if asking about the nutritional intake of a people over the age of 65 did the researchers chose to interview everyone over 65 years or did they interview years or only those over the age of 90 years? Let s look at a real example: Please read this journal paper: Nutrition survey in an elderly population following admission to a tertiary care hospital. N Azad, J Murphy, SS Amos, Topan J. CMAJ Sep 7;161(5): ) The objectives of this study were: a) To determine the prevalence of malnutrition (What) in elderly patients (Who) in tertiary care hospitals (Where), for 5 consecutive months (When) b) Test the sensitivity and specificity for 3 nutrition screening tools E-course Sample Size Unit Page 3

4 Questions about the sample in this paper:? Is it a random sample Consecutively recruited between July and Nov 1996 admitted to 5 different acute-care hospital services = Convenience sample Answer: No, it is not random, it is a convenience sample.? Is this sample representative of the population of elderly patients in tertiary care healthcare centres Answer: it might be or it might not be, we do not know but often bias is introduced when choosing convenience samples. Ideally one should choose a simple random sample if a list of the population do exist. Often it is not the case, so a convenience sample has to be selected instead.? What can I do to minimize bias Answer: As suggested by Boushey et al (2008) you might design a systematic, purposive methodology of recruitment and outline these steps in the report. For example, a concerted effort to select consecutively every accessible person that meets the study criteria will help minimize the volunteerism effect.? How do you get a sample that will give you an answer to the question that is reliable (precise enough) and can be used for evidence based practice? Answer: Calculating the number of patients needed (sample size) to estimate the prevalence of malnutrition (primary outcome variable) with a certain confidence level (typically 95%) and a certain margin of error. This calculation should be performed before launching the experiment. RECAP: you must choose a sample size based on one hypothesis or one research question the most important to the study!? Were there enough people to estimate the prevalence of malnutrition with a certain precision? Answer: Sample size calculation was not reported. 160 patients were included in the study but detailed nutrition assessments were only completed by 152 patients (95% response rate). If we assume true malnutrition prevalence being 60% (based on previous studies) and the estimate provided with a 95% confidence, considering the number of patients with information available (n=152), the malnutrition prevalence estimate of the study was provided with a margin of error of 7.8%, which is more often expressed with a confidence interval of 15.6% width 1. 1 See the formula used for this calculation at the end of document and/or watch P8iQ0 E-course Sample Size Unit Page 4

5 Confidence interval width is twice the margin of error and reveals the uncertainty we have about our estimate. The higher the width of the interval, the more uncertain we are about our estimate. More technical details on how to calculate sample size follow Why Sample Size Matters? Sample size affects the certainty we have about our estimate. The smaller the sample size, the bigger the uncertainty around our estimate. It is always important to calculate your sample size before the experiment is carried out because: 1. If too small, we might not be able to answer the question i.e. we ll estimate with not enough precision or be unable to detect differences between groups, when they really do exist waste of time and money. 2. If too large, might imply putting more patients at risk, when fewer will suffice. Or it will take more time and work of researchers than needed. Purpose of sample size calculation: Estimate a parameter with enough precision To test a hypothesis with enough power? What factors influence sample size when estimating a parameter such as a mean, a proportion Answer: 1. Precision of the estimate (Confidence interval width or margin of error): The more precise the estimate, the smaller the width, the bigger the sample size. 2. Level of confidence: The biggest the confidence the lowest α or error type I. a. α or error type I is the probability of committing an error when making a decision in a hypothesis test. Which error?. Rejecting the null hypothesis when it is actually true. Typically this is a very small probability 5% 3. Variability of the parameter: σ 2 (variance) for quantitative variables such as weight or p(1-p) for qualitative variables such as sex or malnutrition (yes vs. not). Information of this variability might be obtained from a pilot study, previous research or just for qualitative variables, the worse-case scenario might be assumed i.e. p=q=0.5 a. p being the probability that the event of interest occurs i.e. suffering malnutrition ) and q equal to (1-p), the probability of not suffering it. Remember (p+q) always sum up to 1. E-course Sample Size Unit Page 5

6 ? What other factors might influence sample size when comparing two groups and/or testing a hypothesis whose outcome are proportions and means Answer: 1. Whether the hypothesis is unilateral or bilateral 2. Probability of committing error type I or α i.e. 5%. 3. Probability of committing error type II or β i.e. 20% (power=1- β=80%). a. β = Probability of accepting the null hypothesis, when it is actually false. Power: Probability of rejecting the null hypothesis when is actually false. Power equals to 1- β. 4. Minimum clinical difference or association: Size of the difference that is meaningful to detect. 5. Variability of the variable of study in the reference group 6. Other factors: Statistical test, study designs, comparison of more than 2 groups, unequal group size, etc, etc? Still confused try these additional resources on sample size and its basics. The following might help you to prepare and review terms and methods before you answer questions at the end of this unit on sample size. Audiovisual support Confidence Intervals - Confidence intervals - what are they? How to interpret them? (Great for non-native English speakers slow pace + subtitles) (7 min) - Confidence intervals (CI) - Interpretation (5 min) - Confidence intervals around the mean (5 min) - Confidence intervals for population proportions Calculating required sample size to estimate population means Calculating requires sample size to estimate population proportions E-course Sample Size Unit Page 6

7 Confidence interval to estimate a difference of means More on statistics Is there any program that I can freely download to perform the required calculations? Here you have only a few from those available EPIDAT 3.1 Pan American Health Organization (PAHO/OMS) and Health General Direction, Galician Regional Government, Spain. Language: Spanish/English General Description: EPIDAT is a public domain software designed as an epidemiologic calculator for the handling of tabulated data. Data management capabilities are limited. The module tutorial is a great teaching tool on biostatistics and epidemiology including work-out examples and having a focus on critical use of the statistical procedures (tutorial only provided in Spanish). The module Sampling includes: Sample size, Sample Selection, Subjects assignment to treatment Epi Info Version Centers for Disease Control (CDC), USA Language: English/others General Description: Epi Info is a public domain software package designed for the global public health community of practitioners and researchers. It provides for easy questionnaire and database construction, data entry and analysis with epidemiologic statistics, graphs, and maps. The menu utility Statcalc allows sample size estimation for a population proportion, relative risk (unmatched cohort study) and odds ratio (case-control study). PS 3.0: Power and Sample Size Calculation William D. Dupont WD, Plummer, WD, Vanderbilt University, Department of Biostatistics, USA (2009). Language: English General Description: PS is a public domain interactive program for performing power and sample size calculations. It can be used for studies with dichotomous, continuous, or survival response measures. E-course Sample Size Unit Page 7

8 Suggested reading: Publishing Nutrition Research: A Review of Sampling, Sample Size, Statistical Analysis, and Other Key Elements of Manuscript Preparation, Part 2 Carol J. Boushey, PhD, MPH, RD, Jeffrey Harris, DrPH, RD, Barbara Bruemmer, PhD, RD and Sujata L. Archer, PhD, RD Journal of the American Dietetic Association Volume 108, Issue 4, Pages Sample size for beginners Freely available at: Calculating sample size help sheet Freely available at: Things to bear in mind when thinking of sample size to estimate a difference between 2 groups in a grant proposal Source: Publishing Nutrition Research: A Review of Sampling, Sample Size, Statistical Analysis, and Other Key Elements of Manuscript Preparation, Part 2. Carol J. Boushey et al. JADA. Volume 108, Issue 4, Pages (April 2008) E-course Sample Size Unit Page 8

9 Now try these questions 1. Why is important the selected sample to be representative? Circle the wrong one a. To decrease sampling error b. To avoid bias c. To be able to extrapolate sample results to target population d. To ensure our results are valid 2. Why is important to calculate sample size before carrying out a research project? Circle the most comprehensively true answer a. To make sure we ll be able to detect a difference, if that difference does exist b. To avoid treating subject unnecessarily c. To avoid waste of time and money d. All of the above 3. When calculating sample size to estimate a parameter such as the mean, which factors influence the minimum number needed? Circle the most comprehensively true answer a. Confidence level b. Confidence width c. Variance of the variable d. All of the above 4. When calculating sample size to estimate a difference, which factors influence the minimum number needed? Circle the most comprehensively true answer a. Variance of the variable in the reference group b. Confidence level c. Minimum important difference d. Power of the test e. All of the above 5. If not sample size calculated, we might Circle the wrong one a. Have insufficiently precise b. Introduce bias c. Lack of power d. Be unable to reject null hypothesis e. Introduce obscurity on a certain topic Answers: 1(a); 2(d); 3(d); 4(e); 5(b) E-course Sample Size Unit Page 9

10 Annex Formula for the estimation of number of participants needed when the study objective is estimating a proportion with a certain level of confidence and a certain margin of error n: Number of subjects needed z α/2 : Z-value for a probability α/2, if a (1-α) estimation confidence is required i.e. if 95% estimate confidence, then α/2=2.5% or as probability, and the corresponding z-value = 1.96 p: value of the prevalence (proportion, probability) assumed in the population e: margin of error expressed as a probability. Two times e is the width of the confidence interval. It relates to the estimate precision. n= 2 z α/2 p (1-p) 2 e 2 2 (z α/2) (p) (1-p) (1,96 ) (0,6) (0,4) 152= = 2 2 e x 2 2 (1.96 ) (0.6) (0.4) 0,922 x = ; x = = or 7.8% E-course Sample Size Unit Page 10