Review Article Statistical methods and common problems in medical or biomedical science research

Transcription

1 Int J Physiol Pathophysiol Pharmacol 017;9(5): /ISSN: /IJPPP Review Article Statistical methos an common problems in meical or biomeical science research Fengxia Yan 1, Mayberry Robert 1, Yonggang Li 1 Department of Community Health an Preventive Meicine, Morehouse School of Meicine, Atlanta, GA, USA; ICF, Atlanta, GA, USA Receive September, 017; Accepte October 16, 017; Epub November 1, 017; Publishe November 15, 017 Abstract: Statistical thinking is crucial for stuies in meical an biomeical areas. There are several pitfalls of using statistics in these areas involving in experimental esign, ata collection, ata analysis an ata interpretation. This review paper escribes basic statistical esign problems in biomeical or meical stuies an irects the basic scientists to better use of statistical thinking. The contents of this paper were base on previous literatures an our aily basic support work. It inclues the sample size etermination an sample allocation in experimental esign stage, numerical an graphical ata summarization, an statistical test methos as well as the relate common errors at esign an analytic stages. Literatures an our aily support works show that misunerstaning an misusing of statistical concept an statistical test methos are significant problems. These may inclue ignoring the sample size an ata istribution, incorrect summarization measurement, wrong statistical test methos especially for repeate measures, ignoring the assumption for t-test or ANOVA test, failing to perform the ajustment for multiple comparison. This review intens to help the researchers in basic meical or biomeical areas to enhance statistical thinking an make fewer errors in stuy esign an analysis of their stuies. Keywors: Statistical test methos, sample size etermination, ata summarization Introuction Statistical thinking is commonly use in public health, clinical research an community stuies. In biomeical area, the stuies usually involve animals or cell lines an the stuies typically use the experiments which introuce more challenge for ata analysis an statistical metho utilization as compare to other areas of stuies. Biomeical research, incluing stuy planning, experimental esign, sample size an power etermination, ata collection, ata analyses an interpretation as well as manuscript preparation, all require statistical support. While utilization of statistical thinking in biomeical an biological research are iscusse by several researchers [1-5], there are still a lot of misunerstoo an misinterpretation of statistical concepts. In this review paper, we will explain the relate statistical issues in basic science research base on our aily statistical support activities. First, we will explain how to perform the experimental esign incluing sample size etermination an sample allocation. Then, the appropriate escriptive statistical summaries will be illustrate. Lastly, the most efficient statistical inference test methos will be escribe. Experimental esign stage In the experimental esign stage, the major neglects inclue two parts: 1) sample size etermination an ) allocation of the sample to ifferent groups using ranomization an efining the replication rate. Sample size etermination Correctly efining the appropriate sample size is critical for clinical an community stuies as well as the basic biomeical stuies incluing animal stuy. In basic biomeical science, the researchers usually use the sample size that most similar stuies use which woul be a limitation for the stuy. Charan [6] propose metho to calculate the sample size for animal stuy base on power analysis which is very similar to

2 Table 1. Sample size calculation formulas [7] Formula Note Continuous outcome One sample Two inepenent samples with common stanar eviation Two paire samples Categorical outcome One sample v n$ ( Z + Z ) a b 4v n$ ( Z + Z ) a b v n$ ( Z + Z ) a b δ is the etecte ifference σ is the population stanar eviation δ is the relevant ifference in means σ is the population stanar eviation σ is the stanar eviation of the mean ifference δ is the mean ifference ( Z p0(1 - p0) + Z p1(1 - p)) p 1 n $ a b 0 is the success rate uner null hypothesis ( p - p p ) 1 is the success rate uner alternative hypothesis Two inepenent groups p (1 - C pc) + p (1 - p ) p E E C is the success rate in control group n $ ( Za + Zb ) p E is the success rate in treatment group δ is - p E - p C Two paire sample ( Z + Z ) f f is the proportion of iscorant pairs n $ a b is the proportion ifference Where α is type I error level an β is the type II error level Int J Physiol Pathophysiol Pharmacol 017;9(5):

3 the clinical an community stuy sample size etermination. If the research only has fixe number of subjects, then post hoc power analysis nees to be provie to show ability of the statistical tests. Five questions nee to be consiere in avance to etermine the minimum require sample size to test a hypothesis. 1. What is the primary outcome? The primary outcome is the measurement of the main stuy purpose. In basic biomeical science, there may be more than one primary outcome. Sample size can be calculate for each outcome an the most conservative one can be applie as the stuy sample size. The caveat, however, is to make sure the sample size for the primary outcomes is well within this conservative estimate.. How to measure the primary outcome? Is it a continuous variable or a categorical variable? If the primary outcome is a continuous variable, what is the mean an variance in the general population an what is the expecte ifference between the control an intervention group? If the primary outcome is a categorical variable, what is the proportion in control group an what is the expecte ifference between the control group an intervention group? 3. What is the stuy esign an what kin of statistical methos are use for the ata analysis? In the basic biomeical science area, most of the stuies are ranomize-control esigne stuies. Hypothesis testing is involve most of the time. 4. What is the type I error an type II error levels. Type I error is the false positive rate which occurs when the researchers conclue that there was a significant effect when there is none. The most commonly use value for type I error is 5% which means only by chance the researchers may have 5% positive finings. Conversely, type II error is false negative rate which means the researchers fail to make positive conclusions when there are real positive finings. Type II error is use to efine the stuy power which is the ability to etect positive finings when there is any. Appropriate efine sample size can minimize type I error an increase the power. 5. What is the expecte attrition, e.g., eath of animals in the stuy sample. A lot of basic biomeical science, especially animal stuy invol- ve the eath of the animals or cells, or the lost sample which require aitional consieration to ajust the require sample size. For example, if power analysis show that the minimum require sample size is 0 an the stuy expects 10% attrition rate, the final sample size shoul be 0/0.9, which nees 3. The sample size calculation formulas for ifferent statistical tests are liste in Table 1. While the power analysis metho is the most powerful way to etermine the sample size, it is not always possible to have the require information, such as stanar eviation or effect size. A metho calle resource equation metho [8] can be applie in this situation. Here is the formula base on the ecie sample size. E = Total number of animals - Total number of groups Where E is a value to represent if the sample size is optimum. If E is less than 10 it means the sample size is not large enough. If E is greater than 0 it means the sample size is too large. For example, if the researcher has evelope 5 groups to perform the intervention in an animal stuy, the total number of animals shoul be from 15 to 5. This crue metho shoul only be use when sample size calculation cannot be one by power analysis metho. Sample allocation an replication Ranomization is the main principle for sample allocation. Ranomization can greatly reuce the unintentional bias an confouning effects which may exist between the control group an the intervention group. Ranomization ensures later use of probability theory to perform the statistical analysis [5]. It is important to make sure the control an treatment have same conitions in various aspects, such as the time of the ay an temperature. For example, if 0 animals were require to perform the stroke moel, these 0 animals shoul be ranomize into two groups: 10 controls an 10 interventions. If the investigators only can o 4 animals per ay, the morning an afternoon shoul also be consiere for the ranomization. Data summarization Summarizing ata inclues numerical summary an graphical summary. The purpose of ata 159 Int J Physiol Pathophysiol Pharmacol 017;9(5):

4 Table. Description of numerical ata summarization Measure of center Continuous numerical normal Mean Stanar eviation Continuous numerical skewe Meian Interquartile range Nominal categorical No center an sprea, use frequency with percentile Orinal categorical Meian Interquartile range Table 3. Description of graphical ata summarization Variable type Graph type Nominal categorical Bar graph, pie chart Orinal categorical Bar graph, pie chart Continuous numerical Histogram, scatter plot, box-plot Discrete numerical Box-plot, stem-leaf plot summarization is to escribe the center of the ata an how the ata sprea from the center. The ata escription in the first step can show some basic information for the reaers incluing the overall sample size, sample size in each group as well as variable summaries. Appropriate measurements nee to be chosen base on the properties of the variables. For continuous numerical variables, if the sample size is large enough an the ata is normally istribute, mean woul be the best way to represent the center of the ata an stanar eviation will be use to escribe the variables ispersion. The most incorrectly use concept in basic science area is the stanar error of the mean (SEM) [9] which equals sample stanar eviation ivie by square root of the sample size. The SEM measures the variation of ifferent sample mean while the stanar eviation measures the variation of the current sample. The purpose of using SEM is to construct confience interval for the current sample mean or to perform statistical test. Marcel [10] escribe that all original journals publishe in 01 in Cariovascular research, Circulation, Heart Failure an Circulation Research ha inappropriate use of SEM. In the same paper, the author state that basic science stuies ha a 7.4-fol higher level of inappropriate use of SEM compare to clinical stuies. For continuous numerical variables, if the sample size is not large enough to etermine the ata istribution or the ata istribution is not normal, the meian with interquartile range (IQR) woul be a better measure to escribe the variable compare to the mean with stan- ar eviation. For Measure of sprea orinal categorical variables, such as the Likert ata or isease stage ata, the meian with IQR woul be a goo way to escribe. Frequency with per- centile will be use to escribe the categorical variables. Another point of the ata summarization relate to the later statistical test metho is, for example, if the statistical metho the researcher chosen for the ata analysis is nonparametric metho, the meian with IQR shoul be the correct way to summarize ata instea of using mean with stanar eviation. Different ways of summarization methos were liste in Table. Graphics can provie informatively isplay of the variables. Bar charts or pie charts are usually use for categorical variables. Histogram can be use to escribe continuous variables an show the istribution an potential outlier of the variables. Scatter plot with regression line often will be use for two continuous variables. Box-plot woul be use to escribe the orinal categorical variables or non-normal istribute continuous variables. If the researcher chose to use meian an IQR to escribe ata an use nonparametric statistical test later, the box-plot woul be the correct figure to use. All these figures can show the istribution of the ata an make intuitively comparison between groups or over time perios. The escription of graphical summarization was liste in Table 3. Statistical analysis tests The stuy esign, research hypothesis, the type of the variables as well as the ata istribution efines the statistical analytical tests. The major tests metho an relate variable types were escribe in Table 4. In basic meical or biomeical research, researchers often make mistakes in the following aspects: 1) two sample t-test an ANOVA, ) repeate measurements, 3) non-parametric test, an 4) multiple comparisons. Two sample t-test an ANOVA In basic meical science area, continuous measurements are the most common outcomes, 160 Int J Physiol Pathophysiol Pharmacol 017;9(5):

5 Table 4. The basic statistical test methos for ifferent type of variables Outcome Assumption Statistical test Numerical ata One sample mean Normal istribution One sample t-test One sample meian Not normally istribute One sample meian test or sign test/signe rank test Two inepenent means Normal istribution Two sample t-test Two inepenent means Not normally istribute Wilcoxon-Mann-Whitney test (Wilcoxon rank sum test) Two correlate means Normal istribution Paire t-test Two correlate means Not normally istribute Wilcoxon signe rank test Inepenent more than two means Normal istribution ANOVA test Inepenent more than two means Not normally istribute Kruskal Wallis test Correlate (or repeate) more than two means Normal istribution Repeate measure ANOVA Relationship between two numerical variables Normal istribution Pearson correlation test Relationship between two numerical variables Not normally istribute Spearman correlation test Categorical ata One proportion test Binomial test Relationship between two categorical variables Chi-square test Relationship between two categorical variables, but one or more cells have expecte value less than 5 Fisher s exact test Test same categorical outcome on matche pairs McNemar test Binary outcome measure repeately Repeate measure logistic regression 161 Int J Physiol Pathophysiol Pharmacol 017;9(5):

6 such as protein, DNA an RNA measurements. Two samples t-test is the way to compare two sample means an ANOVA woul be the best way to compare more than two group means. There are two assumptions for t-test: the ata shoul be normally istribute an ientical inepenent. There are three assumptions for ANOVA: the ata shoul be normally istribute with inepenent measurements an groups have equal variances. Several methos can be use to examine if the ata is normally istribute. The graphic illustration is not goo for small sample size. The most commonly use graphic methos are Q-Q plot an P-P plot. Statistical test is more accurate to test the normality which involves several ifferent methos like K-S test, or S-M test. For basic science stuy, the sample size 5-6 is not enough to perform the normality test using any of these tests because small sample size is not large enough to provie enough power to o the test [7]. Repeate measurements In basic biomeical science area, measurements of same subjects in repeate times are very common stuy esign features. If the stuy esign is the pre-post esign an the outcome is continuous variable, then paire t-test shoul be applie instea of two inepenent sample t-test. If the outcome is categorical variable, the McNemar test woul be the correct test to consier the inner correlation. If the time perios are more than two an the research woul like to examine the outcome changes following ifferent time perios, the correct statistical test shoul be one-way repeate measure ANOVA. Some investigators chose two-way ANOVA to consier the time effect which is not correct because the measurements in ifferent time points are correlate which violate with the inepenent assumption of ANOVA test. If other risk factors may have an effect on the outcome, mix can be applie. Non-parametric test If the unerlying istribution is not normal or the sample size is not large enough to etermine the unerlying istribution, then nonparametric test shoul be consiere. Nonparametric tests o not epen on the unerlying istribution which woul be more robust compare to the parametric test. But, because the non-parametric tests ignore the original ata value an perform the test base on the rank, some information will be lost which makes the non-parametric tests have less power compare to the relate parametric test. The researchers nee to choose a goo balance point between parametric an non-parametric test. In basic biomeical research, small sample size is a common problem ue to several factors incluing buget consieration. A small sample size makes it ifficult to etermine the unerlying istribution for most parametric tests. Relate non-parametric methos can be use. For example, if we woul like to compare one specific protein level between control an treatment groups an we only have 5 sample for each group, non-parametric test calle Wilcoxonrank-sum test can be performe instea of two sample t-test. Multiple comparisons Multiple tests nee to be performe if the researcher has multiple groups to compare or if the research has multiple outcomes to examine between the same group. For example, ANOVA test is to test group means for more than two groups comparison. If one is intereste in more than two comparisons, there is a multiple comparisons problem which requires the multiple comparisons ajustment. For genetics stuy, multiple gene expressions between groups will be examine which also involves multiple tests. When multiple tests are conucte, the overall alpha level cannot be applie to all performe tests. There are a number of ways to perform the multiple comparisons ajustment. Bonferroni ajustment is to use propose type I error level ivie by the total number of tests which is the most conservative an easiest metho. For example, if 5% type I error was chosen for the significance level an there are total of 5 tests, then 0.05/5=0.01 woul be the new significance level. Multiple comparison ajustment can reuce the type I error which enlarges our test power. For example, 100 genes were teste between control an treatment group, there will be 5 positive expresse genes only ue to the type I error if significance level of 5% was use. There are several ways to perform the ANOVA post hoc test [11]. Tukey s HSD is goo for allpairwise comparisons while the Dunnett s proceure is appropriate for many-to-one comparisons because the Dunnett s proceure only 16 Int J Physiol Pathophysiol Pharmacol 017;9(5):

7 consiers k-1 tests (k is the comparison group number) whereas Tukey s HSD assumes k*(k-1) tests which increases the false positive rate. Another metho calle Fisher s least significant ifference proceure (LSD) you can choose for three groups pairwise comparison even though Meier state that LSD has potential loss of power. Main pairwise comparisons for non-parametric Kruskal-Wallis test inclue Dunnett s proceure an Bonferroni ajustment. Dunn s z-test statistics approximates exact rank-sum test statistics by using the mean ranking of the outcome in each group from preceing Kruskal- Wallis test. Some researchers choose to use Mann-Whitney test for each two groups an perform Bonferroni ajustment which is acceptable whereas it may introuce more bias ue to using ifferent mean rank values from the Kruskal-Wallis test. False iscovery rate (FDR) was wiely use for genetics stuy with multiple comparison ajustment. Conclusion Statistical thinking is critical in basic biomeical research area incluing stuy planning, sample allocation, ata escription, ata analysis an interpretation. Investigators from ifferent areas of basic science alreay notice the importance of applying correct statistical thinking into their research. This review may help the basic science researchers to unerstan some basic statistical concepts to avoi significant an routine errors in ata summarization an statistical test methos for simple stuy esigns. If the basic science researcher s stuy involves a complex stuy esign, we encourage the researcher to consult with the statistician from the stuy conceptualization phase to better process the stuy. Acknowlegements The project escribe was supporte by the National Institute on Minority Health an Health Disparities (NIMHD) an National Institute of Allergy an Infectious Diseases (NIAID) Grant Number U54MD007588, a component of the National Institutes of Health (NIH) an its contents are solely the responsibility of the authors an o not necessarily represent the official views of NIMHD, NIAID or NIH, an The project escribe was supporte by the National Center for Avancing Translational Sciences of the National Insti- tutes of Health uner Awar Number UL1TR The content is solely the responsibility of the authors an oes not necessarily represent the official views of the National Institutes of Health. Disclosure of conflict of interest None. Aress corresponence to: Dr. Fengxia Yan, Department of Community Health an Preventive Meicine, Morehouse School of Meicine, 70 Westview Dr. SW, Atlanta, GA 30310, USA. Tel: ; Fax: ; fyan@msm. eu References [1] Binu VS, Mayya SS, Dhar M. Some basic aspects of statistical methos an sample size etermination in health science research. Ayu 014; 35: [] Sprent P. Statistics in meical research. Swiss Me Wkly 003; 133: 5-9. [3] Ali Z, Bhaskar SB. Basic statistical tools in research an ata analysis. Inian J Anaesth 016; 60: [4] Bajwa S. Basics, common errors an essentials of statistical tools an techniques in anesthesiology research. J Anaesthesiol Clin Pharmacol 015; 31: [5] Sullivan LM, Weinberg J, Keaney JF Jr. Common Statistical Pitfalls in Basic Science Research. J Am Heart Assoc 016; 5. [6] Charan J, Kantharia ND. How to calculate sample size in animal stuies? J Pharmacol Pharmacother 013; 4: [7] Rosner B. Funamentals of biostatistics, seventh eition. Boston: Brooks/Cole: Cengage Learning; 011. [8] Festing MF, Altman DG. Guielines for the esign an statistical analysis of experiments using laboratory animals. ILAR J 00; 43: [9] Bare MP, Bare PJ. What to use to express the variability of ata: Stanar eviation or stanar error of mean? Perspect Clin Res 01; 3: [10] Wullschleger M, Aghlmani S, Egger M, Zwahlen M. High incorrect use of the stanar error of the mean (SEM) in Original Articles in Three Cariovascular Journals Evaluate for 01. PLoS One 014; 9: e [11] McHugh ML. Multiple comparison analysis testing in ANOVA. Biochem Me (Zagreb) 011; 1: Int J Physiol Pathophysiol Pharmacol 017;9(5):