Use of the Quantitative-Methods Approach in Scientific Inquiry. Du Feng, Ph.D. Professor School of Nursing University of Nevada, Las Vegas

Transcription

1 Use of the Quantitative-Methods Approach in Scientific Inquiry Du Feng, Ph.D. Professor School of Nursing University of Nevada, Las Vegas

2 The Scientific Approach to Knowledge Two Criteria of the Scientific Approach Logical Empirical Fundamental Basis of Social/Behavioral Sciences Theory Research Method Statistics

3 Video: Battling Bad Science by Ben Goldacre ata Good science? Bad science? - Research method makes the call

4 The Social Science Research Process The research problem - interest, idea The hypothesis - theory The research design research method, population & sampling Measurement conceptualization & operationalization Data collection - observations Data analysis - data processing, analysis Generalization & Application

5 The research journey authored by Stephanie Fleischer SAGE publications Ltd 2014

6 Anatomy of A Research Article Title Abstract Introduction Methods Results Discussion References Appendix

7 Elements of a Research Proposal Specific Aims - What do you intend to do Significance - Why is the work important? Background - Literature Review Subjects - How are you going to select your sample? Measurement (of the IV s and DV s) Data-collection methods (research design) Data analysis plan Timeline Budget

8 Asking Testable Questions The first step in conducting scientific research Not all questions are testable In order to be testable, research questions should be empirically grounded, precise, specific, and value-free Identify the independent and dependent variables DEVELOPMENT-WB-MASTER/research-question/

9 The variable language Variables: logical groupings of attributes gender; age Attributes: characteristics male, female; young, old

10 Deciding the level of numerical measurement authored by Stephanie Fleischer SAGE publications Ltd 2014

11 Criteria for Evaluating Social and Behavioral Scientific Research Internal Validity: To what extent does the research design permit us to reach causal conclusions about the effect of the IV on the DV? External Validity: To what extent can we generalize from the research sample and setting to the population and settings specified in the research hypothesis?

12 Internal Validity Threats to Internal Validity Attrition History Instrumentation Maturation Selection Statistical regression Testing effects Interaction effect

13 Maximize Internal Validity The Goal of Experiments Random assignment of individuals to treatment conditions means that confounding variables are equally distributed across conditions As such, confounding variables are unlikely to be responsible for observed differences between treatment conditions

14 External Validity Threats to External Validity Reactive effect of testing Interaction of selection and treatment Reactive effects of experimental arrangements Multiple-treatment interference Biased sample

15 Maximize External Validity - Random and Nonrandom Sampling The extent to which the results of a study generalize to the population of interest To be confident about such a generalization, the sample must be representative of the population of interest

16 Figure 7.1 Population, sample and individual cases authored by Stephanie Fleischer SAGE publications Ltd 2014

17 Two Ways to Get a Sample Probability Sampling Every element of the population has a known nonzero probability of being selected Random selection is used at some point in the process Nonprobability Sampling Something else. Bottom Line: Nonprobability sampling makes it impossible to estimate sampling errors With nonprobability sampling, judgments about external validity are rarely on firm ground.

18 Measurement Thinking about Total Variability - If X = T + E, then: var (X) = var (T) + var (E)

19 Scales (Levels) of Measurement Nominal Ordinal Interval Ratio - An example: Age measured at the nominal level: 1=young; 2=old measured at the ordinal level: 1=young; 2=middle aged; 3=old 1=infant; 2=child; 3=adolescent; 4=adult measured at the interval level: years of age

20 Quality of Measurement Reliability - the extent to which a measuring technique would yield consistent results when applied repeatedly - More technical: To what extent do observed scores reflect true scores? - Less technical: How consistent is the assessment? Validity - the extent to which a measuring technique adequately reflects the real meaning of the concept under consideration - Am I measuring what I intend to measure?

21 Reliability Coefficients Reliability coefficients reflect the proportion of true score variance to observed score variance r xx var( T) var( X ) Therefore reliabilities range from 0.0 (no true score variance) to 1.0 (all true-score variance)

22 Various Types of Reliability Internal Consistency (Content) Random error affects responses to items on an assessment Test-Retest (Time) The construct stays the same. However, random errors vary from one occasion to the next. Inter-Rater (Observer Biases)

23 Various Types of Validity Content Criterion-related Construct Concurrent Convergent Discriminant criterion-related (more data-based) construct (general evidence-gathering) content (more theory-based)

24 Research Design Experiments vs Surveys Cross-sectional vs longitudinal The ultimate goal: - To establish causality

25 Experimental Approach to Studying Causal Relations Intervention and Control Random assignment of participants to two or more conditions of an experiment The Independent variable is the variable that is manipulated or the experimental conditions (e.g., treatment or notreatment) The Dependent variable is the outcome of interest in the particular study (observed/measured)

26 Design 1: Randomized Two-Group Design Treatment Outcome (DV) Pool of Participants Control Outcome (DV)

27 Design 2: Pretest-Posttest Two-Group Design Pretest Treatment Posttest (DV: Post - Pre) Pool of Participants Pretest Control Posttest (DV: Post - Pre)

28 Design 3: Solomon Four-Group Pretest Treatment Posttest Pretest Control Posttest Treatment Posttest Control Posttest

29 Clinical Trials Explanatory trials - aim to test whether an intervention works under highly controlled and optimal situations - High internal validity Pragmatic trials - designed to evaluate the effectiveness of interventions in real-life routine practice conditions - High external validity

30 In the ideal world A representative sample obtained by random sampling A sufficiently large N RCT Valid and reliable measurement No missing or incomplete data No attrition in longitudinal studies High fidelity of intervention studies

31 Challenges Relating To Research Design Random sampling is infeasible -> sample bias -> low external validity Difficult to recruit participants Random assignment of recruited participants is in feasible - > low internal validity Hard to measure outcome variables Poor measurement quality, measurement error Missing data due to drop-outs Incomplete data due to refusal to answer certain questions Other challenges

32 Prospective Power Analysis The purpose is to determine the minimum sample size required for achieving a desirable power for a specific hypothesis test, for obtaining a confidence interval with a specified width, to estimate a parameter with a maximum error of estimation for a stated probability

33 Inputs for Power Analyses The desired power Effect size (or, statistics that can be used to calculate effect size) Cohen s d, Hedges g: standardized mean or mean difference ω 2, Cohen s f, or f 2 : the variance explained regression slope, odds ratio, etc. The α level One-sided or two-sided test The number of in independent variables (and correlations between them) The number of groups based on each independent variable Distributional properties (e.g., standard deviation) of the random variables in the analytical model Standard deviation of the residuals (e.g., for linear regression) The number of occasions Bivariate correlation between repeated measurements (e.g., for repeated measure ANOVA) etc.

34 Software for Power Analysis Stand-alone power/specialized programs for power analysis only G*Power FREE Pass SPSS/Sample Power General purpose statistical packages SAS Stata SPSS faculty pack R FREE Software for multilevel modeling PINT FREE Optimal Design Software FREE MLPowsim FREE Mplus

35

36 Some learning sources Jacob Cohen (1992), "A power primer", Psychological Bulletin 112 (1): UCLA Statistical Computing Seminars: Introduction to Power Analysis: Intermediate Power Analysis: s_-_intermediate_course_for_ucla_white.pdf Advanced Power Analysis: s_-_advanced_course_for_ucla_white.pdf

37 Statistical Analysis The goal of statistical analysis is hypothesis testing Research strategies (including type of research design, sampling strategy, temporal design of measurements, measurement of expected outcomes, manipulation or measurement of the independent variables, etc.) should be guided by theory Statistical models should be consistent with theoretical reasoning and chosen research methods

38 Choosing appropriate statistical tests Clearly stated specific, empirically grounded, and testable hypothesis Numbers of IVs and DVs Type (level of measurement) of each IV/DV Continuous variables are measured at the interval or ratio levels Discrete variables are measured at the nominal or ordinal levels A dichotomous variable is discrete, but can be treated as a continuous variable statistically

39 How to communicate with your statistician?

40 How to communicate with your statistician? ~continued

41

42 Variables How many? What type? Analysis Online text Movies Group Differences: Comparing Group Means DV -> One -> Continuous IV One IV One IV One Discrete-two independent groups Discrete-three or more independent groups Two time points (pre- and post-tests), or two dependent groups IV One Three or more time points Repeated measure ANOVA IV Two or more IV Two or more Discrete; between-subject factors only Discrete and continuous; between-subject factors only Independent t-test Statsoft StatsLecture Andy Field One-way ANOVA Statsoft StatsLecture Andy Field Paired t-test U of M Paired t-test using SPSS PowerPoint StatsLecture Andy Field Factorial ANOVA PowerPoint StatsLecture Andy Field ANCOVA Andy Field Andy Field IV Two or more Two or more time points & at least one between-subject factor ( all discrete) Mixed model ANOVA Mixed Model ANOVA/ ANCOVA Andy Field IV Two or more Two or more time points & at least one between-subject factor (discrete and continuous) Mixed model ANCOVA Mixed Model ANOVA/ ANCOVA Same as above DV-> Two or more -> Continuous IV Any number All discrete MANOVA MANOVA/ MANCOVA Andy Field IV Any number Discrete and continuous MANCOVA MANOVA/ MANCOVA Factorial MANCOVA

43 Variables How many? What type? Analysis Online text Movies Relationships Among Variables DV One Continuous IV One Continuous Pearson s r Hypothesis testing with Pearson s r IV Two or more Continuous and/or dichotomous DV Two or more Continuous IV Two or more Continuous Canonical correlation or path analysis StatsLecture Hypothesis testing with Pearson s r Multiple regression Statsoft MR_ SPSS (Brief) MR_ SPSS (more depth) Introduction Canonical Correlation DV Multiple Continuous IV None Exploratory factor analysis IV Latent variables Confirmatory factor analysis DV One Discrete IV One Discrete 2 test for contingency tables Statsoft CFA StatsLecture Andy Field CFA CFA using Amos StatsLecture IV Two or more Continuous Discriminant analysis Statsoft DFA using SPSS DFA IV Two or more Continuous, discrete, and dichotomous Logistic regression Overview Logistic regression

44 Which statistical analysis would be appropriate for the following research example? Suppose that you are interested in whether students learn better in the morning or in the afternoon. You recruited 20 students and taught them vocabulary words at 7 am one day and at 2pm another day.

45 Which statistical analysis would be appropriate for the following research example? Suppose that you have designed an intervention program to help people with addiction to alcohol stop drinking. In order to evaluate the program, you selected 30 subjects and randomly assigned 15 of them to receive the intervention, and the other 15 subjects to the control group. After the intervention, you measure the number of drinks each person has in a week, and you want to know if the two groups are different in terms of drinking.

46 Which statistical analysis would be appropriate for the following research example? Suppose you are interested in whether a training program improves nurses skills in identifying sepsis. Two groups of nurses are randomly assigned to an experimental group who receive the training (n=50), and a control group (n=50) who do not receive the training. Then, both groups are asked to evaluate a simulation case which involves sepsis. It is observed whether the nurses successfully identify sepsis with the hypothetical patient.

47 More research examples from you? Write down a clear statement of your specific research hypothesis Identify the independent variable(s) and dependent variable(s) of the hypothesis, Identify the type of each of the IV(s) and DV(s) as being either discrete (or, categorical) or continuous.

48 Can data lie?

49 Videos: When is reproducibility an ethical issue? by Keith Baggerly dayworkshops/13w5083/videos/watch/ Baggerly.mp4

50 Background of the Case

51 Data Management Entering Data Cleaning Data Scoring Processing data

52 Documentation Codebook Annotated forms A notebook of everything: proposal, description of the project, codebook, annotated forms, instrument and their scoring instructions, consent forms, data error reports and responses, all copies of communications, etc.

53 Software Packages R & Rstudio SAS SPSS STATA

54 R and RStudio Free R ( RStudio ( From the very basic to very sophisticated Examples of descriptive statistics and plots A Shiny Apps example

55

56

57 United States Health.Expenditure: Life.Expectancy: Region:North America Population:

58 United States Health.Expenditure: Life.Expectancy: Region:North America Population:

59

60 Questions? Thank you!