Use of the Quantitative-Methods Approach in Scientific Inquiry Du Feng, Ph.D. Professor School of Nursing University of Nevada, Las Vegas
The Scientific Approach to Knowledge Two Criteria of the Scientific Approach Logical Empirical Fundamental Basis of Social/Behavioral Sciences Theory Research Method Statistics
Video: Battling Bad Science by Ben Goldacre http://www.ted.com/playlists/130/the_dark_side_of_d ata Good science? Bad science? - Research method makes the call
The Social Science Research Process The research problem - interest, idea The hypothesis - theory The research design research method, population & sampling Measurement conceptualization & operationalization Data collection - observations Data analysis - data processing, analysis Generalization & Application
The research journey authored by Stephanie Fleischer SAGE publications Ltd 2014
Anatomy of A Research Article Title Abstract Introduction Methods Results Discussion References Appendix
Elements of a Research Proposal Specific Aims - What do you intend to do Significance - Why is the work important? Background - Literature Review Subjects - How are you going to select your sample? Measurement (of the IV s and DV s) Data-collection methods (research design) Data analysis plan Timeline Budget
Asking Testable Questions The first step in conducting scientific research Not all questions are testable In order to be testable, research questions should be empirically grounded, precise, specific, and value-free Identify the independent and dependent variables https://webcampus.unlv.edu/bbcswebdav/courses/oe_nurs-780- DEVELOPMENT-WB-MASTER/research-question/
The variable language Variables: logical groupings of attributes gender; age Attributes: characteristics male, female; young, old
Deciding the level of numerical measurement authored by Stephanie Fleischer SAGE publications Ltd 2014
Criteria for Evaluating Social and Behavioral Scientific Research Internal Validity: To what extent does the research design permit us to reach causal conclusions about the effect of the IV on the DV? External Validity: To what extent can we generalize from the research sample and setting to the population and settings specified in the research hypothesis?
Internal Validity Threats to Internal Validity Attrition History Instrumentation Maturation Selection Statistical regression Testing effects Interaction effect
Maximize Internal Validity The Goal of Experiments Random assignment of individuals to treatment conditions means that confounding variables are equally distributed across conditions As such, confounding variables are unlikely to be responsible for observed differences between treatment conditions
External Validity Threats to External Validity Reactive effect of testing Interaction of selection and treatment Reactive effects of experimental arrangements Multiple-treatment interference Biased sample
Maximize External Validity - Random and Nonrandom Sampling The extent to which the results of a study generalize to the population of interest To be confident about such a generalization, the sample must be representative of the population of interest
Figure 7.1 Population, sample and individual cases authored by Stephanie Fleischer SAGE publications Ltd 2014
Two Ways to Get a Sample Probability Sampling Every element of the population has a known nonzero probability of being selected Random selection is used at some point in the process Nonprobability Sampling Something else. Bottom Line: Nonprobability sampling makes it impossible to estimate sampling errors With nonprobability sampling, judgments about external validity are rarely on firm ground.
Measurement Thinking about Total Variability - If X = T + E, then: var (X) = var (T) + var (E)
Scales (Levels) of Measurement Nominal Ordinal Interval Ratio - An example: Age measured at the nominal level: 1=young; 2=old measured at the ordinal level: 1=young; 2=middle aged; 3=old 1=infant; 2=child; 3=adolescent; 4=adult measured at the interval level: years of age
Quality of Measurement Reliability - the extent to which a measuring technique would yield consistent results when applied repeatedly - More technical: To what extent do observed scores reflect true scores? - Less technical: How consistent is the assessment? Validity - the extent to which a measuring technique adequately reflects the real meaning of the concept under consideration - Am I measuring what I intend to measure?
Reliability Coefficients Reliability coefficients reflect the proportion of true score variance to observed score variance r xx var( T) var( X ) Therefore reliabilities range from 0.0 (no true score variance) to 1.0 (all true-score variance)
Various Types of Reliability Internal Consistency (Content) Random error affects responses to items on an assessment Test-Retest (Time) The construct stays the same. However, random errors vary from one occasion to the next. Inter-Rater (Observer Biases)
Various Types of Validity Content Criterion-related Construct Concurrent Convergent Discriminant criterion-related (more data-based) construct (general evidence-gathering) content (more theory-based)
Research Design Experiments vs Surveys Cross-sectional vs longitudinal The ultimate goal: - To establish causality
Experimental Approach to Studying Causal Relations Intervention and Control Random assignment of participants to two or more conditions of an experiment The Independent variable is the variable that is manipulated or the experimental conditions (e.g., treatment or notreatment) The Dependent variable is the outcome of interest in the particular study (observed/measured)
Design 1: Randomized Two-Group Design Treatment Outcome (DV) Pool of Participants Control Outcome (DV)
Design 2: Pretest-Posttest Two-Group Design Pretest Treatment Posttest (DV: Post - Pre) Pool of Participants Pretest Control Posttest (DV: Post - Pre)
Design 3: Solomon Four-Group Pretest Treatment Posttest Pretest Control Posttest Treatment Posttest Control Posttest
Clinical Trials Explanatory trials - aim to test whether an intervention works under highly controlled and optimal situations - High internal validity Pragmatic trials - designed to evaluate the effectiveness of interventions in real-life routine practice conditions - High external validity
In the ideal world A representative sample obtained by random sampling A sufficiently large N RCT Valid and reliable measurement No missing or incomplete data No attrition in longitudinal studies High fidelity of intervention studies
Challenges Relating To Research Design Random sampling is infeasible -> sample bias -> low external validity Difficult to recruit participants Random assignment of recruited participants is in feasible - > low internal validity Hard to measure outcome variables Poor measurement quality, measurement error Missing data due to drop-outs Incomplete data due to refusal to answer certain questions Other challenges
Prospective Power Analysis The purpose is to determine the minimum sample size required for achieving a desirable power for a specific hypothesis test, for obtaining a confidence interval with a specified width, to estimate a parameter with a maximum error of estimation for a stated probability
Inputs for Power Analyses The desired power Effect size (or, statistics that can be used to calculate effect size) Cohen s d, Hedges g: standardized mean or mean difference ω 2, Cohen s f, or f 2 : the variance explained regression slope, odds ratio, etc. The α level One-sided or two-sided test The number of in independent variables (and correlations between them) The number of groups based on each independent variable Distributional properties (e.g., standard deviation) of the random variables in the analytical model Standard deviation of the residuals (e.g., for linear regression) The number of occasions Bivariate correlation between repeated measurements (e.g., for repeated measure ANOVA) etc.
Software for Power Analysis Stand-alone power/specialized programs for power analysis only G*Power FREE Pass SPSS/Sample Power General purpose statistical packages SAS Stata SPSS faculty pack R FREE Software for multilevel modeling PINT FREE Optimal Design Software FREE MLPowsim FREE Mplus
Some learning sources Jacob Cohen (1992), "A power primer", Psychological Bulletin 112 (1): 155 159. http://www.unt.edu/rss/class/mike/5030/articles/cohen1992.pdf http://web.vu.lt/fsf/d.noreika/files/2011/10/cohen-j-1992-a-power-primer-kokio-reikiaimties-dyd%c5%beio.pdf http://classes.deonandan.com/hss4303/2010/cohen%201992%20sample%20size.pdf UCLA Statistical Computing Seminars: Introduction to Power Analysis: http://www.ats.ucla.edu/stat/seminars/intro_power/ Intermediate Power Analysis: http://www.ats.ucla.edu/stat/seminars/power_analysis/power_analysi s_-_intermediate_course_for_ucla_white.pdf Advanced Power Analysis: http://www.ats.ucla.edu/stat/seminars/power_analysis/power_analysi s_-_advanced_course_for_ucla_white.pdf
Statistical Analysis The goal of statistical analysis is hypothesis testing Research strategies (including type of research design, sampling strategy, temporal design of measurements, measurement of expected outcomes, manipulation or measurement of the independent variables, etc.) should be guided by theory Statistical models should be consistent with theoretical reasoning and chosen research methods
Choosing appropriate statistical tests Clearly stated specific, empirically grounded, and testable hypothesis Numbers of IVs and DVs Type (level of measurement) of each IV/DV Continuous variables are measured at the interval or ratio levels Discrete variables are measured at the nominal or ordinal levels A dichotomous variable is discrete, but can be treated as a continuous variable statistically
How to communicate with your statistician?
How to communicate with your statistician? ~continued http://pubs.acs.org/doi/abs/10.1021/om4000067
Variables How many? What type? Analysis Online text Movies Group Differences: Comparing Group Means DV -> One -> Continuous IV One IV One IV One Discrete-two independent groups Discrete-three or more independent groups Two time points (pre- and post-tests), or two dependent groups IV One Three or more time points Repeated measure ANOVA IV Two or more IV Two or more Discrete; between-subject factors only Discrete and continuous; between-subject factors only Independent t-test Statsoft StatsLecture Andy Field One-way ANOVA Statsoft StatsLecture Andy Field Paired t-test U of M Paired t-test using SPSS PowerPoint StatsLecture Andy Field Factorial ANOVA PowerPoint StatsLecture Andy Field ANCOVA Andy Field Andy Field IV Two or more Two or more time points & at least one between-subject factor ( all discrete) Mixed model ANOVA Mixed Model ANOVA/ ANCOVA Andy Field IV Two or more Two or more time points & at least one between-subject factor (discrete and continuous) Mixed model ANCOVA Mixed Model ANOVA/ ANCOVA Same as above DV-> Two or more -> Continuous IV Any number All discrete MANOVA MANOVA/ MANCOVA Andy Field IV Any number Discrete and continuous MANCOVA MANOVA/ MANCOVA Factorial MANCOVA
Variables How many? What type? Analysis Online text Movies Relationships Among Variables DV One Continuous IV One Continuous Pearson s r Hypothesis testing with Pearson s r IV Two or more Continuous and/or dichotomous DV Two or more Continuous IV Two or more Continuous Canonical correlation or path analysis StatsLecture Hypothesis testing with Pearson s r Multiple regression Statsoft MR_ SPSS (Brief) MR_ SPSS (more depth) Introduction Canonical Correlation DV Multiple Continuous IV None Exploratory factor analysis IV Latent variables Confirmatory factor analysis DV One Discrete IV One Discrete 2 test for contingency tables Statsoft CFA StatsLecture Andy Field CFA CFA using Amos StatsLecture IV Two or more Continuous Discriminant analysis Statsoft DFA using SPSS DFA IV Two or more Continuous, discrete, and dichotomous Logistic regression Overview Logistic regression
Which statistical analysis would be appropriate for the following research example? Suppose that you are interested in whether students learn better in the morning or in the afternoon. You recruited 20 students and taught them vocabulary words at 7 am one day and at 2pm another day.
Which statistical analysis would be appropriate for the following research example? Suppose that you have designed an intervention program to help people with addiction to alcohol stop drinking. In order to evaluate the program, you selected 30 subjects and randomly assigned 15 of them to receive the intervention, and the other 15 subjects to the control group. After the intervention, you measure the number of drinks each person has in a week, and you want to know if the two groups are different in terms of drinking.
Which statistical analysis would be appropriate for the following research example? Suppose you are interested in whether a training program improves nurses skills in identifying sepsis. Two groups of nurses are randomly assigned to an experimental group who receive the training (n=50), and a control group (n=50) who do not receive the training. Then, both groups are asked to evaluate a simulation case which involves sepsis. It is observed whether the nurses successfully identify sepsis with the hypothetical patient.
More research examples from you? Write down a clear statement of your specific research hypothesis Identify the independent variable(s) and dependent variable(s) of the hypothesis, Identify the type of each of the IV(s) and DV(s) as being either discrete (or, categorical) or continuous.
Can data lie? http://pss.sagepub.com/content/22/11/1359.abstract
Videos: When is reproducibility an ethical issue? by Keith Baggerly http://www.birs.ca/events/2013/5- dayworkshops/13w5083/videos/watch/20130814112 1-Baggerly.mp4
Background of the Case http://www.nature.com/news/2011/110111/full/469139a/box/1.html
Data Management Entering Data Cleaning Data Scoring Processing data
Documentation Codebook Annotated forms A notebook of everything: proposal, description of the project, codebook, annotated forms, instrument and their scoring instructions, consent forms, data error reports and responses, all copies of communications, etc.
Software Packages R & Rstudio SAS SPSS STATA
R and RStudio Free R (http://cran.r-project.org) RStudio (https://www.rstudio.com) From the very basic to very sophisticated Examples of descriptive statistics and plots A Shiny Apps example http://shiny.rstudio.com/gallery/google-charts.html
United States Health.Expenditure:3747.692121 Life.Expectancy:75.62195122 Region:North America Population:266278000
United States Health.Expenditure:8607.875672 Life.Expectancy:78.64146341 Region:North America Population:311591917
Questions? Thank you!