Principles of Measurement Measurement As a way of understanding, evaluating, and differentiating characteristics Provides a mechanism to achieve precision in this understanding, the extent or quality As a basis for setting standard, making decision or making conclusion The ability to measure a variable, no matter how indirectly, is dependent on the definition Criteria for assigning values and units to variables must be systematic, objectivity Some rules of measurements are obvious and easy to learn. Others require rigorous training and practice for the instruments to be applied efficiently The exactness of a measure is referred to as the precision Directness of Measurement Direct measurement Few measurements (variables) are measured directly Do you think temperature is measured directly? We do not observe temperature! Indirect measurement Direct observation to infer a value for a more abstract phenomenon Sensitivity Simply not available: abstract constructs which are defined as a function of many interrelated concepts 1
Levels of Measurement Nominal: discrete categories, mutually exclusive, Gender, ethnic background Ordinal: rank ordering of the values, not equal distance between values e.g. Income range, disease stage, likert scales Interval: continuous, equal numeric distance between equal length intervals Does not provide the absolute amount of attribute because of the absence of zero on the interval e.g. A temperature of 0 degree does not represent an absence of temperature Ratio: continuous measurement, has an absolute zero point that has empirical meaning e.g. weight (because of the existence of absolute zero, justify to say 50 kg doubles 100 kg), age, force Multiple Measures of an Abstract Concept Locating existing instruments Evaluate existing instruments Assess readability and levels of measurements Reliability Concern with how consistently the measurement technique measures the concept of interest and free from error Many reliability coefficients are based on measures of correlation Correlation indicates how sets of measurements vary together, but does not tell the extent of the agreement between them 2
Measurement Error The difference between the true value and the observed value Systematic error: Predictable errors of measurements Source of bias, Constant and therefore do not present a problem for reliability regression towards the mean: extreme values tend to move closer to the average Random error: Error due to chance Affects accuracy of measurement Many sources of error can be attributed to inaccuracy of instruments, variability of the characteristic being measured, rater differences, Can be minimized by careful planning, clear operational definitions, and inspection of instruments / equipment Stability: concerned with the consistency of repeated measures or testretest reliability Test-retest reliability: testing effects, rater bias, test-retest intervals, intra-rater reliability Equivalence: consistency between two versions of the same instrument (alternate form reliability) or two observers (inter-rater reliability) Homogeneity: correlation among items within an instrument, internal consistency, determined by Cronbach s alpha coefficient, split-half reliability 3
Validity The extent to which an instrument reflects the concept being examined Face validity: appears valid Content validity: Adequacy with which the universe is sampled by a test, representative of the whole, no statistical index to assess content validity (Waltz and Bausell s Index of content validity CVI, 1981) Readibility (The Fog Index of readibility) Validity from contrasting groups (Hagerty and Patusky s sense of belonging instrument tested among community college, patients with major depression, and retired Roman catholic nun) Criterion-related validity: validity from examining convergence (correlation with established instruments/gold standard); concurrent (two tests implement at the same time / predicting concurrent event) and predictive validity (establish a measure which will be a valid predictor of some future criterion score / event) Validity from examining divergence (negative correlation with an instrument measuring a totally opposite construct) Known group method: test the instrument and compare with the expected results Validity from discriminant analysis (test the discriminant power of the instrument) Successive verification of validity / cross validation Multitrait-Multimethod correlation: Measures reflect similar underlying phenomenon as the one under test will yield similar results and hence highly correlated. Those measure different traits produce low or negative correlations 4
Screening Many measuring instruments are designed as screening tools Sensitivity: The test s ability to detect the true positive (detect the disease correctly) Specificity: The test s ability to detect the true negative (conclude the absence of disease correctly) A highly specific instrument rarely tests positive if a person does not have the disease. Measurement Strategies Physiologic measures: e.g. blood pressure, ECG, blood glucose, creatinine clearance, Objective direct measurements Observations: unstructured observations or structured (checklist, categorization, rating); Direct or indirect Interviews: unstructured or structured, need training or pilot testing, probing and recording Questionnaires: open-ended, closed-ended, do it in person/ phone/ mail/ selfadministered 5
Scales: Verbal (mild, moderate, severe, very severe) Numeric (no pain 0 ------------------ 10 a great deal of pain) Likert scale (disagree strongly 1 --------------- 5 agree strongly) Semantic differential e.g. men with tattoos are: traditional 1 2 3 4 5 6 7 progressive wise 1 2 3 4 5 6 7 foolish Visual analog scale (worst possible pain ----------------------- No pain) 5 cm Development: Define concepts, design scales, item review, preliminary test, item analyses, refine instrument, validity check, reliability check, compile norms on scale, publish result, further test in fields, refine if necessary Delphi technique 1. To measure judgement of experts, assess priority or making forecasts 2. Questionnaires were sent and results were summarized for another round of opinion 3. Feedback on the consensus of the group tends to centralize opinion Conclusion could be misleading. Projective techniques Assumption: Response under unstructured and ambiguous situations reflects attitude, desires, personality, and motives. Inferences are made on the meaning of activities or behaviours under the mentioned situations. E.g Interpreting children s painting Diaries 6
7