Measurement 500 Research Methods Mike Kroelinger
Levels of Measurement Nominal Lowest level -- used to classify variables into two or more categories. Cases placed in the same category must be equivalent. The categories must be exhaustive -- all persons or items must fit into one of the categories. Must also be mutually exclusive -- one person or item can't fit more than one category. Examples race and gender.
Levels of Measurement Ordinal Numbers only used to indicate the rank order of cases of a variable. Cannot measure or evaluate the different in value between each case. No mathematical or statistical operations (you can't add label 1 to label 2, etc.). Examples: Hardness of metal Personnel evaluations of performance
Levels of Measurement Interval Has all of the above characteristics with the added requirement of equal distances or intervals between labels Represent equal distances in the variables of your study. Examples: Foot-candle levels in lighting Differences in air temperature
Levels of Measurement Ratio Has all of above features plus an absolute zero point. Enables you to multiple and divide scale numbers to create ratios between labels. Examples: Income ranges Number of years of school Age in years
Best Rule of Thumb Measure at the highest level possible!
Reliability A matter of whether a particular technique, applied repeatedly to the same object, would yield the same result each time. Good example would be weight scales. Another would be an architect's scale vs. a ruler.
Reliability Reliability does not insure accuracy any more than precision insures it. Changing your bathroom scales to let you weigh less is an example -- called bias. Light meters another example. Reliability infers stability and consistency. Is your operational definition measuring something consistently and dependably? Repeats of measure yield consistent results.
Reliability A highly unreliable measure cannot be valid! A very reliable measure may still not be valid because you could be consistently measuring something other than what you intended to measure.
Types of Reliability Test-Re-test Repeat of the measures -- have respondents complete questionnaire more than once. Observers record the same behavior more than once.
Types of Reliability Parallel Two similar or alternative forms of a measure are designed to be as similar as possible. Administered successively to the same group of subjects -- high correlation between scores of the measurements would mean high reliability.
Types of Reliability Split-half Always a good idea to make more than one measurement of a variable. As example -- a questionnaire with ten items that measure a variable (prejudice). With split half, you would randomly assign the items to two sets of five subjects. If the two sets of items measure the variable differently you would have reliability problems in measuring the variable.
Types of Reliability Internal Consistency Item by item examination of relationships attempts to determine to what extent the items measure the same concept. Also referred to as homogeneity. Determined statistically.
Types of Reliability Research-worker (Babbie) Measurement reliability can be caused by research workers. Example -- coding the way interview questions are asked. Ways to solve include extensive training, practice, and cross-checking For example, a supervisor calls a subset of the S's and repeats a portion of the interview. Verifies information by replication.
Validity Validity refers to the extent to which an empirical measure adequately reflects the real meaning of the concept being studied. Validity cannot be assessed directly.
Validity What we want to know is the "goodness of fit" between an operational definition and the concept that it supposedly measures. In short, your operational definition should truly reflect what the concept means! If you are measuring what you intend to measure, you have validity.
Validity Excellent example is amniocentesis -to determine gender of unborn children. Valid because it is almost perfect in accuracy of prediction. Not-valid example is old wife's tales about childbirth and gender. Second example - scales for weighing yourself vs. a subjective estimate of how much you weigh.
Types of Validity Face validity An empirical measurement may or may not match common agreements and our individual ideas of a particular concept. For example, we may measure a person's morale by the number of grievances that they file with the union. We could not measure morale through the number of books the S's read during off-duty hours! The second measure would not have face validity.
Types of Validity Criterion-related validity Sometimes called predictive validity; is based on an external criterion. An example is the college board exam -- validity is shown through its ability to predict college success of students.
Types of Validity Content Validity Refers to the degree to which a measure covers the range of meanings included with the concept. A good example is a math test -- will not show range of meanings if it covers addition alone -- must cover subtraction, multiplication, division.
Types of Validity Construct Validity Based on the way a measure relates to other variables within a conceptual framework or model. Emphasizes the meaning of responses to your instrument. Is the instrument measuring the intended concept or can it be interpreted as measuring something else?
Types of Validity Construct Validity Good example would be marital satisfaction gift buying predicts satisfaction. If satisfied and dissatisfied spouses were equally likely to buy each other gifts, you would not have validity of your measure.
Summary A valid measure is reliable but a reliable measure may not be valid! Measure at the highest possible level! Use prior studies whenever possible to select measures for your research. Make sure that you select studies that have reliability and validity. The more they are used, the better we define their reliability and validity!
Questions