Quantitative Research Methods Sofia Ramström Medicinska vetenskaper, Örebro Universitet Diagnostikcentrum, klinisk kemi, Region Östergötland Disposition I. What is science and what is quantitative science? II. How do we measure? III. How do we determine the correctness of our measurement? What is good science and research? Science what it is Science is a way of thinking much more than it is a body of knowledge (Carl Sagan) A way of obtaining knowledge by means of objective observations (McBurney & White) A body of techniques for investigating phenomena and acquiring new knowledge, as well as for correcting and integrating previous knowledge. It is based on observable, empirical, measurable evidence, and subject to laws of reasoning. Basic assumptions of science A true, physical universe exists Although there may be randomness and thus unpredictability in the universe, it is primarily an orderly system The principles of this universe can be discovered, particularly through scientific research Knowledge of the universe is always incomplete. New knowledge can, and should, alter current ideas and theories. Therefore, all knowledge and theories are tentative Inductive and deductive logic Deduction ("top-down" approach) Induction ("bottom up" approach) 1
The goals of science The discovery of regularities Description Discovering laws (a statement that certain events are regularly associated with each other in an orderly fashion) Search for causes Development of Theories Theory (a statement or set of statements explaining one or more laws, usually including one indirect concept required to explain the relationship) Theories must be falsifiable Categories of research Quantitative vs. Qualitative Quantitative research: studies that make use of statistical analyses to obtain their findings. Qualitative research: do not attempt to quantify their results through statistical summary or analysis (Interviews and observations, case studies.) Quantitative science Propose specific hypotheses as explanations of natural phenomena Design experimental studies that test these predictions for accuracy. These steps are repeated in order to make increasingly dependable predictions of future results. Build theories that bind more specific hypotheses together into logically coherent wholes. This in turn aids in the formation of new hypotheses. Quantitative Methods Process of inquiry must be objective. The researcher must make complete documentation of data and methodology available for careful scrutiny by other scientists and researchers. This also allows statistical measures of the reliability of the results to be established. Attempt to achieve control over the factors involved in the area of inquiry, which may in turn be manipulated to test new hypotheses in order to gain further knowledge. Cornerstones of scientific methods Empirical evidence (empiricism) Logical reasoning (rationalism) Sceptical attitude (scepticism) about presumed knowledge, being undogmatic (willing to change one's beliefs) The Scientific Method Observation Description Prediction Control Falsifiability Causal explanation 2
Hypotheses An educated and testable guess about the answer to your research question Each hypothesis must make a prediction These predictions are then tested, and the hypotheses can either be supported or refuted on the basis of the data Falsifiability- elimination of plausible alternatives A gradual process that requires repeated experiments by multiple researchers who must be able to replicate results All hypotheses and theories are in principle subject to disproof There is a point at which there might be a consensus about a particular hypothesis or theory, yet it must in principle remain tentative. The hypothetico-deductive method Demands falsifiable hypotheses (null hypothesis), framed in such a manner that the scientific community can prove them false with a certain agreed probability If the null hypothesis is refuted by a certain probability, the hypothesis is not necessarily proven, but remains provisional Causal explanation Requirements generally regarded as important to scientific understanding: Identification of the causes of a particular phenomenon. Covariation of events. The hypothesized causes must correlate with observed effects. Time-order relationship.the hypothesized causes must precede the observed effects in time. Serendipity Discoveries that are unanticipated, fortuitous, or lucky Appear to be stumbled on while the scientist is looking for something else. Once discovered they stimulate new theories and/or research Seredipitous findings are not happy accidents. They could easily have been missed had the scientist not been alert to the implication of the observation. Such alertness requires both a prepared mind and a real sense of quriosity. II. How do we measure? What do you measure in your research? What is your current hypothesis? How do you need to design your study to be able to prove your hypothesis? 3
Measurement Measurement is the assignment of numbers to objects or events in a systematic fashion. a set of operations having the object of determining a value of a measurable quantity Measurement The set of values that can result from the appropriate application of a particular measurement/analytical procedure is called possible values. The measured quantity must be expressed with both a value and a unit Variables Aspect of a testing condition that can change or take on different characteristics with different conditions Dependent variable The dependent variables are those that are observed to change in response to the independent variables Independent variable The independent variables are those that are deliberately manipulated to invoke a change in the dependent variables Controlled variable Condition or variable intentionally kept constant throughout the study The purpose is to minimize the effects of the controlled variable on the dependent variables 4
Randomized variable Usually a variable that cannot be controlled outside of the control of the experimenter Its influence on the dependent variables is made equal by randomizing into treatment groups Population Placebo sample Treatment sample Confounding factors Factor(s) influencing the results of the study that are not influencing the groups in the same way and therefore make the conclusions uncertain Confounded variables Confounded variables vary with the independent variable, and their effects on a dependent variable cannot be distinguished from the effects of the independent variable Such a relation between two observed variables is termed a spurious relationship Confounding is a major threat to the validity of inferences made about cause and effect, i.e. internal validity, as the observed effects should be attributed to the confounder rather than the independent variable. Makes the study of the independent variable itself more difficult since more or less of the observed effect may be due to effect of the confounded variable Minimize the effect(s) of confounded variables Keep them the same and constant for all groups Remove them, e.g. by extraction/purification of a chemical sample Randomize the subjects to the different treatment groups Adjust statistically for the presence of the confounder(s) when analyzing the data Spurious relation A situation in which measures of two or more variables are statistically related (they cover) but are not in fact causally linked usually because the statistical relation is caused by a third variable. 5
Spurious relation. Stimulus range and dose-response If a clearcut relation between the stimulus (independent factor) and the effect (dependant factor) can be established, it increases the probability that the independent factor is causally related to the dependent factor(s) From: http://tylervigen.com/ Choosing levels of the independent variable Levels of the independent influence the results The stimulus should cover as much of the range as possible They should be close enough to prevent overlooking interesting effects In within-subjects studies, at least seven stimuli should be presented if possible If the continuum is quantitative, the stimuli should be logarithmically spaced Choosing levels of the independent variable test wide range and sufficient number Choosing levels of the independent variable- spacing stimuli carefully III. How do we determine the correctness of our measurement? Can you estimate what would be the next value measured in one of your experiments? Why/why not? If anyone else would try to repeat your experiment, would they get the same results? Why/why not? 6
Replication Are others able to replicate/repeat your results? Use replicates to get a better estimate of the mean and variation o A true replicate means a true study of a new member of the population o Measuring a characteristic of ONE member of the population many times is not replication (but is important to determine the uncertainty of your method) Repeatability and reproducibility Repeatability - Variation between results when repeating the same measurement under the exact same conditions (within a short time interval) Reproducibility Variation between results when measuring the same thing (sample) at different conditions (and times) Controls Control treatment Subjects treated in the same manner as the treated group with the exception of the substance under study Sham treatment Control of the possible effect of a treatment necessary in addition to the treatment with the active drug or control. E.g. oophorectomy operation with and without removing the ovaries Random error The divergence, due to chance alone, of an observation on a sample from the true population value Leading to lack of precision in the measurement of association Three major sources Individual biological variation Sampling variation Measurement variation Random error Random error can never be completely eliminated since we can study only a sample of the population. Random error can be reduced by the careful measurement of exposure and outcome Sampling error can be reduced by optimal sampling procedures and by increasing sample size Systematic error = bias Results that differ systematically from the true results A study with small systematic error has high trueness A study with small systematic and random error has high accuracy 7
( riktighet ) Reliability (of a measurement or of a study) The property of a measurement that gives the same result on different occasions ( noggrannhet ) ( precision ) Related to Good experimental design Absence or minimum of confounding factors Low measurement uncertainty From: Menditto A, Patriarca M, Magnusson B. Understanding the meaning of accuracy, trueness and precision. Accred Qual Assur 2007;12:45-47 Validity (of a measurement) Validity is cumulative The property of a measurement that tests what it is supposed to test / measures what it is supposed to measure The best available approximation to the truth Internal validity External validity Construct validity Statistical validity Metrology and statistics The use of metrological principles ensure that the measurement techniques used are clearly understandable, comparable and repeatable by others Eurachem/CITAC Guide http://www.eurachem.org/ Statistics help researchers minimize the likelihood of reaching an erroneous conclusion about the relationship between the variables being studied 8
ISO quality concepts ISO quality concepts Accuracy Trueness True value Accuracy ( noggrannhet ) Compliance with the true value including precision and accuracy Precision Trueness ( riktighet ) Compliance with the true value Precision ( precision ) Consistency of a series of determinations High accuracy, Low precision Low accuracy, but high precision..! http://www.mathsisfun.com/accuracy-precision.html The hourglass structure of research Begin with broad quesions Narrow down and focus in on a particular topic Operationalize Measure/Observe Analyse data Reach conclusions Generalize back to questions Fundamentals of good quantitative research Relevant and intelligent hypothesis Systematic approach the researcher processes logically through steps in a pre- defined plan Adherence to sound metrological principles Samples representative of the population = randomization, stratification etc. Control minimizing bias, random variation and confounding factors Replication Repeating observations in your own study Most observations where random variation is high Others try to replicate your results under other conditions The scientific method is not a recipe It requires intelligence, imagination, and creativity. It is an ongoing cycle, constantly developing more useful, accurate and comprehensive models and methods. 9
Take home message YOU are the one knowing your experiments best, so If you have formed a hypothesis YOU are the one that should make every effort to try to disprove it If you can t do this It is likely to be true Thank you for your attention and good luck with your future research! Acknowledgements to: Elvar Theodorsson Jan Gillquist 10