DATA is derived either through. Self-Report Observation Measurement

Size: px

Start display at page:

Download "DATA is derived either through. Self-Report Observation Measurement"

Arron Sherman
5 years ago
Views:

1 Data Management

2 DATA is derived either through Self-Report Observation Measurement

3 QUESTION ANSWER DATA DATA may be from Structured or Unstructured questions? Quantitative or Qualitative? Numerical or Categorical? Continuous or Discrete? AS IS or a Derivative?

4 Pre-Data Collection Review Data Collection Forms /Methods Consider the following criteria- If QUANTITATIVE VALIDITY (internal) VALIDITY (external) RELIABILITY (objectivity) If QUALITATIVE CREDIBILITY TRANSFERABILITY DEPENDABILITY CONFIRMABILITY

5 Pre-Data Collection Review Data Collection Forms Internal Validity how much you are able to prove what you want to prove (is often connected to design of study) External Validity how much applicable your findings are to subjects outside of your sample (is often connected to sampling method) Reliability how precise methods are

6 Reliability and Validity of Instruments Reliability = Consistency Validity = Accuracy

7 Reliability may mean three different things depending on how data was derived - Stability : Measurements Internal Consistency : Self Reports Equivalence : Observations

Stability extent to which scores are similar on 2 separate administrations of an instrument Test retest reliability Pearson s r Correlation Coefficient Appropriate for relatively enduring attributes

8 Stability extent to which scores are similar on 2 separate administrations of an instrument Test retest reliability Pearson s r Correlation Coefficient Appropriate for relatively enduring attributes Change wording of questions in a functionally equivalent form, or simply to change the order of the questions Alternate/ Equivalent form reliability Pearson s r Correlation Coefficient helps to overcome the "practice effect"

9 r = 0.95 r = Positive Correlation Negative Correlation

10 Strength of Linear Association perfect linear relationship strong relationship ( ) 0.5 moderate relationship ( ) weak relationship ( ) no linear relationship

11 Internal consistency extent to which all the instrument s items are measuring the same attribute Split-half technique Cronbach s Alpha, & Spearman- Brown Coefficient Appropriate for most multi-item instruments Coefficient alpha Cronbach s Alpha only

12 Cronbach s alpha <0.6 Unacceptable reliability Acceptable reliability Good reliability Very Good reliability >0.95 Items redundant

13 Equivalence degree of similarity between multiple raters/observers using an instrument Interobserver or interrater reliability Cohens Kappa and Multirater Kappa (nominal) Most relevant for structured observations degree of similarity between alternate forms used by raters/observers using an instrument Intra-observer McNemar Statistic (binary data) Most relevant for structured observations

14 Kappa <0 Excellent agreement Very good Good Fair Slight Poor No agreement

15 Validity degree to which an instrument measures what it is supposed to measure

16 Validity Face Content Criterionrelated Construct Measurements/questions related to variable/concept degree by which items are representative of characteristic compare to gold standard related to instruments of same characteristic and not others (measures what is suppose to be measured) subjective judgement Expert validation using Content Validity Index Use criteria and calculate validity coefficient Use factor analysis

17 Sampling how you got your subjects/respondents Sampling Probability or Random Sampling equal probability or there is predictable possibility Non Probability or non random Sampling not everyone has chance

18 Probability Sampling Simple random sampling Stratified random sampling Cluster sampling Systematic sampling

19 Multi Stage Sampling Use a combination of methods Mainly utilizes cluster sampling Used for large area sampling

20 Methods of Non-random Sampling Convenience (and/or volunteer) sampling Accessible samples Snowball sampling referral Purposive sampling

21 Types of Purposive Sampling Maximum variation sampling get extremes Homogenous sampling get similar cases Extreme (deviant) case sampling get one extreme Intensity sampling get representative for each intensity Typical case sampling Critical case sampling Criterion sampling Theory based sampling Quota Sampling Sampling confirming/disconfirming cases

22 Terminology (Not the same) Random Sampling method of selection Random Assignment Required in experiments

23 Eligibility Criteria Inclusion Criteria Exclusion Criteria

24 Considerations that Affect Sample Size in Quantitative Studies Homogeneity of the population more homogenous less sample Effect size (strength of relationships) greater effect size to be achieved, more sample needed Attrition (loss of subjects) more samples Interest in subgroup analyses - more samples Sensitivity of the measures ability to detect difference; more sensitive, more samples

25 Sample Size Estimation Equations and methods exist to estimate a theoretical sample size needed to prove a certain effect

26 Pre-Data Collection Review Data Collection Forms Presentable Easy to fill-up Not too long, not too short Proper format & instructions Easy to understand Correct language & vocabulary Retraceable and sufficient info

27 Data Collection Proper Avoid bias Design Bias Internal/External Validity Regression effect Attrition effect Measurement Bias Sampling Bias Procedural Bias Type 3 error (wrong problem)

28 Data Collection Proper Check notes & questionnaires Lacking information Correct information

29 Data Collection Proper Make data retraceable Number Form or Questionnaire Number Respondent Number Control Number List / Sign-up sheets Ask for name or unique identifier

30 Post Data Collection Coding manual

31 Post Data Collection Coding manual Variable Names Variable Labels

32 Post Data Collection Coding manual Qualitative: Encode as is Nominal: Encode each as separate variables or Encode as one variable but each with assigned codes Ordinal: code according to rank Numerical: code as measured

Post Data Collection Coding manual Encode as is: Words or phrases Encode as assigned: Binary Responses Categorical Answers & Multiple Choice Cafeteria Questions Encode

33 Post Data Collection Coding manual Encode as is: Words or phrases Encode as assigned: Binary Responses Categorical Answers & Multiple Choice Cafeteria Questions Encode as ranked/ordered: ordinal Likert Scale Semantic Differential Scale Encode as measured: Rating Scale Visual Analog Scale Bio-physiologic Measures Derived/computed variables

34 Post Data Collection Coding Data Bio-physiologic Measures: check units

35 Post Data Collection Coding Data Use color

36 Post Data Collection Coding Data Centralized less errors, but takes longer Decentralized faster but more prone to errors

37 Post Data Collection Coding Data Vertical vs Horizontal Most statistical softwares read data horizontally; thus if encoded vertically, you need to transpose data

38 Post Data Collection Coding Data Formatting vs No Formatting Formatting helps to avoid mistakes but is not recognized by statistical softwares and may cause some errors.

39 Post Data Collection Coding Data Order as to the questionnaire

40 Post Data Collection Coding Data Post Encoding Checking Count Check frequencies Total/Sum Check for blank or zero information Direction of questions

ADMS Sampling Technique and Survey Studies

ADMS Sampling Technique and Survey Studies Principles of Measurement Measurement As a way of understanding, evaluating, and differentiating characteristics Provides a mechanism to achieve precision in this understanding, the extent or quality As