PÄIVI KARHU THE THEORY OF MEASUREMENT
|
|
- Phebe McCarthy
- 6 years ago
- Views:
Transcription
1 PÄIVI KARHU THE THEORY OF MEASUREMENT
2 AGENDA 1. Quality of Measurement a) Validity Definition and Types of validity Assessment of validity Threats of Validity b) Reliability True Score Theory Definition and Types of reliability Assessment of reliability 2. Levels of Measurement 2
3 AGENDA 1. Quality of Measurement a) Validity Definition and Types of validity Assessment of validity Threats of Validity b) Reliability True Score Theory Definition and Types of reliability Assessment of reliability 2. Levels of Measurement 3
4 Quality of Measurement - Validity - Definition and Types of validity WHAT IS MEASUREMENT? Measurement is the process of observing and recording the observations that are collected as part of a research effort. 4
5 Quality of Measurement - Validity - Definition and Types of validity VALIDITY QUESTIONS ARE CUMULATIVE. Conclusion Internal Is the relationship causal? Is there a relationship between the cause and effect? Construct Did you measure what you wanted to measure? External How well can you generalize from your sample to other persons, place, times? Validity The best available approximation of the truth of a given proposition, inference or conclusion. 5
6 Quality of Measurement - Validity - Definition and Types of validity VALIDITY TYPES CONSTRUCT VALIDITY An assessment of how well your actual programs or measures reflect your ideas or theories Is the operationalization* a good reflection of the construct? Does the operationalization* behave the way it should? TRANSLATION VALIDITY CRITERION-RELATED VALIDITY Face Validity Content Validity *)act of translating a construct into its manifestation Predictive Validity Concurrent Validity Convergent Validity Discriminant Validity 6
7 Quality of Measurement - Validity - Definition and Types of validity TRANSLATION VALIDITY Face Validity you look at the operationalization and see whether "on its face" it seems like a good translation of the construct Content Validity you essentially check the operationalization against the relevant content domain for the construct 7
8 Quality of Measurement - Validity - Definition and Types of validity CRITERION-RELATED VALIDITY Predictive Validity we assess the operationalization's ability to predict something it should theoretically be able to predict Concurrent Validity we assess the operationalization's ability to distinguish between groups that it should theoretically be able to distinguish between Convergent Validity we examine the degree to which the operationalization is similar to (converges on) other operationalizations that it theoretically should be similar to Discriminant Validity we examine the degree to which the operationalization is not similar to (diverges from) other operationalizations that it theoretically should be not be similar to 8
9 Quality of Measurement - Validity - Definition and Types of validity Land of Observation Land of Theory CONSTRUCT VALIDITY Theory Cause construct What you think Cause-effect Construct Effect construct Can we generalize to the constructs? Program What you do Program-outcome Relationship Observations What you see Observation
10 Quality of Measurement - Validity - Definition and Types of validity CONSTRUCT VALIDITY Convergent correlations should be > Discriminant correlations They work together: Evidence for both Evidence for construct validity Convergent Validity Discriminant Validity To establish convergent validity, you need to show that measures that should be related are in reality related To establish discriminant validity, you need to show that measures that should not be related are in reality not related 10
11 Quality of Measurement - Validity - Definition and Types of validity CONSTRUCT VALIDITY WHY? The more complex your theoretical model, the more evidence you are providing that you know what you re talking about! Convergent correlations should be > Discriminant correlations They work together: Evidence for both Evidence for construct validity Convergent Validity Discriminant Validity To establish convergent validity, you need to show that measures that should be related are in reality related To establish discriminant validity, you need to show that measures that should not be related are in reality not related 11
12 GREAT! This analysis provided evidence for both convergent and discriminant validity. It shows that the three self-esteem measures reflect the same construct and locus-of-control measures reflect the same construct and that these two sets of measures reflect two different constructs. BUT: What are the constructs measuring?! How do you show that your measures are actually measuring self-esteem or locus-ofcontrol? 12
13 Quality of Measurement - Validity - Assessment of Construct Validity CONSTRUCT VALIDITY ASSESSMENT Assessment tools: The Nomological Network The Multitrait-Multimethod matrix (MTMM) Pattern Matching Structural Equation Modeling (SEM) 13
14 Quality of Measurement - Validity - Assessment of Construct Validity THE NOMOLOGICAL NETWORK Developed by Lee Cronbach and Paul Meehl (1955) Nomological network=lawful network This network includes Theoretical framework for what you are trying to measure Empirical framework for how you are going to measure it A specification of the linkages among and between those two frameworks Not practical Only useful as a philosophical foundation for construct validity 14
15 Quality of Measurement - Validity - Assessment of Construct Validity THE NOMOLOGICAL NETWORK 15
16 Quality of Measurement - Validity - Assessment of Construct Validity THE MULTITRAIT-MULTIMETHOD MATRIX (MTMM) Developed by Campbell and Fiske (1959) An attempt to provide a practical methodology Introduced convergent and discriminant validity as subcategories of construct validity A matrix of correlations arranged to facilitate the assessment of construct validity Assumes that you have measured each construct (trait) with different methods in a fully crossed design (traits by methods) A restrictive methodology 16
17 Quality of Measurement - Validity - Assessment of Construct Validity MTMM Example: MTMM for 3 concepts Correlations 3 kinds of shapes 1. Diagonals 2. Triangels 3. Blocks 3 Concepts Traits A, B, C 3 Methods 1, 2,
18 Quality of Measurement - Validity - Assessment of Construct Validity INTERPRETATION OF THE MTMM Coefficients in the reliability diagonal should consistently be the highest in the matrix Coefficients in the validity diagonals should be significantly different from zero and high enough to warrant further investigation A validity coefficient should be higher than values lying in its column and row in the same heteromethod block A validity coefficient should be higher than all coefficients in the heterotrait-monomethod triangles 18
19 19
20 Quality of Measurement - Validity - Assessment of Construct Validity PROS AND CONS OF MTMM + - An operational methodology for assessing construct validity A rigorous framework for assessing construct validity Requirement of a fully crossed measurement design Judgemental nature hindering adoption Impossible to quantifiy the degree of construct validity in a study 20
21 Quality of Measurement - Validity - Assessment of Construct Validity PATTERN* MATCHING Pattern* Matching is the of construct validity Degree of correspondence between the theoretical and the observed pattern *)any arrangement of objects or entities; nonrandom and at least potentially 21 describable.
22 Quality of Measurement - Validity - Assessment of Construct Validity A PATTERN MATCHING EXAMPLE 22
23 Quality of Measurement - Validity - Assessment of Construct Validity PROS AND CONS OF PATTERN MATCHING + - Higher generality and flexibility that MTMMs: it does not require that you measure each construct with multiple methods Treatment of convergence and discrimination as continuum Estimation of the overall construct validity for a set of measures in a specific context Specification of what you think about the constructs Requires the precise specification of the theory of the constructs Requires the qualification of both patterns Requires the description in matrices that have the same construct 23
24 Quality of Measurement - Validity - Threats of Validity THREATS TO CONSTRUCT VALIDITY Design threats Inadequate preoperational explication of constructs Mono-Operation Bias single version of your independent variable Mono-Method Bias single method of measurement Interaction of different treatments not able to isolate effects of your program from other treatments Social Threats Hypothesis Guessing participants base behavior on what they guess what the real purpose of the study is Evaluation Apprehension participants afraid of being evaluated Experimenter Expectancies researcher communicates what the desired outcome is Interaction of testing and treatment pretest makes participants more sensitive to the treatment Restricted generalizability across construct side effects of your treatments Confounding constructs and Levels of Constructs dosage level changes results 24
25 AGENDA 1. Quality of Measurement a) Validity Definition and Types of validity Assessment of validity Threats of Validity b) Reliability True Score Theory Definition and Types of reliability Assessment of reliability 2. Levels of Measurement 25
26 Quality of Measurement - Reliability - True Score Theory TRUE SCORE THEORY Assumes that every observation is composed of two components: true value plus random error Error can be divided further into two subcomponents Random error Systematic error 26
27 Quality of Measurement - Reliability - True Score Theory MEASUREMENT ERROR Random error Caused by random factor Pushes scores up or down randomly Random errors sum up to 0 Adds variability to data but does not affect the average performance of the group Systematic error Caused systematically by a certain factor Does effect the whole sample Pushes scores consistently either up or down 27
28 Quality of Measurement - Reliability - Definition and Types of reliability DEFINITION AND TYPES Reliability Repeatability or consistency of a measure Var(T)/Var(X) We cannot directly calculate reliability because we cannot measure the true score component We can estimate the true score component as the covariance or the correlation between two observations of the same measure Cov(X1,X2) It ranges between 0 and 1 (0 r 1) A measure is perfectly reliable (r=1) when the random error equals 0 28
29 Quality of Measurement - Reliability - Definition and Types of reliability RELIABILITY TYPES Inter-Rater or Inter- Observer Reliability Degree to which different raters/observers give consistent estimates of the same phenomenon Used to test how similarly people categorize and score items Test-Retest Reliability Consistency of a measure from one time to another Good tests have less retest variation over time Parallel-Forms Reliability Consistency of two tests constructed in the same way from the same content domain Evaluates different questions that seek to assess the same construct Consistency of results across items within a test Evaluates how consistent the results are for different items for the same construct within a measure Internal Consistency Reliability 29
30 Quality of Measurement - Reliability - Assessment of reliability INTERNAL CONSISTENCY RELIABILITY Consistency of results across items within a test Different approches: Split-Half Reliability Average Interitem Correlation Average Item-Total Correlation Cronbach's Alpha ( 30
31 Quality of Measurement - Reliability - Assessment of reliability SPLIT-HALF RELIABILITY Randomly divides items that measure the same construct into two sets Entire instrument is administered to a sample of people and the total scores for each half is calculated Then the correlation between the two total scores is calculated split half reliability 31
32 Quality of Measurement - Reliability - Assessment of reliability AVERAGE INTER-ITEM CORRELATION Compares correlations between all pairs of questions that measure the same construct by calculating the mean of all paired correlations Average Item-total correlation Takes the average interitem correlations and calculates a total score for the 6 items (seventh variable), then averages these. 32
33 Quality of Measurement - Reliability - Assessment of reliability CRONBACH'S ALPHA Calculates an equivalent to the average of all possible split-half correlations; but you re not calculating it that way Computation is quicker; the most used estimate 33
34 AGENDA 1. Quality of Measurement a) Validity Definition and Types of validity Assessment of validity Threats of Validity b) Reliability True Score Theory Definition and Types of reliability Assessment of reliability 2. Levels of Measurement 34
35 Quality of Measurement - Reliability Levels of measurement LEVELS OF MEASUREMENT Helps in deciding how to interpret data from a variable Defines the kind of statistical analysis Data is classified with different levels of precision or levels of measurement The higher the better 35
36 Quality of Measurement - Reliability Levels of measurement LEVELS OF MEASUREMENT Nominal Attributes are just named No ordering of the attributes (categories) is implied Example: jersey numbers in basketball Ordinal Attributes can be rank-ordered Distances between attributes do not have any meaning Example: grades at school Interval The distance between attributes does have a meaning Example: temperature (in Fahrenheit) Ratio There is always an absolute zero that is meaningful You can construct a meaningful fraction (or ratio) Example: weight 36
37 Quality of Measurement - Reliability Levels of measurement VALIDITY AND RELIABILITY Validity: Quality of Operationalization Reliability: Quality of Measurement 37
38 38 EXERCISES
39 EXERCISE 1. Gender 2. Hair colour 3. Puls rate (in bpm) 4. Body temperatur 5. Team number 6. Shoe size 39
40 EXERCISE 40
41 HOW DO I REMEMBER ALL THIS? IT S EASY! The 5 C s of Validity The DPEFT Validity NoNeMuMuPaMaSEM Construct Validity TIROPI Reliability SACA Reliability Assessment NOIR Levels of Measurement 41
42 THE 5 C S OF VALIDITY Concurrent validity Construct validity Content validity Convergent validity Criterion-related validity 42
43 THE DPEFT VALIDITY Discriminant validity Predictive validity External validity Face validity Translation validity 43
44 NONEMUMUPAMASEM CONSTRUCT VALIDITY ASSESSMENT TOOLS o o o o The Nomological Network The Multitrait-Multimethod matrix (MTMM) Pattern Matching Structural Equation Modeling (SEM) 44
45 TIROPI RELIABILITY Test-Retest Reliability Inter-Rater or Inter-Observer Reliability Parallel-Forms Reliability Internal Consistency Reliability 45
46 SACA RELIABILITY ASSESSMENT Split-half correlation Average interitem correlation Cronbach's alpha Average item-total correlation 46
47 NOIR LEVELS OF MEASUREMENT Nominal Ordinal Interval Ratio 47
Measurement is the process of observing and recording the observations. Two important issues:
Farzad Eskandanian Measurement is the process of observing and recording the observations. Two important issues: 1. Understanding the fundamental ideas: Levels of measurement: nominal, ordinal, interval
More informationADMS Sampling Technique and Survey Studies
Principles of Measurement Measurement As a way of understanding, evaluating, and differentiating characteristics Provides a mechanism to achieve precision in this understanding, the extent or quality As
More information11-3. Learning Objectives
11-1 Measurement Learning Objectives 11-3 Understand... The distinction between measuring objects, properties, and indicants of properties. The similarities and differences between the four scale types
More informationConstruct Validation of Direct Behavior Ratings: A Multitrait Multimethod Analysis
Construct Validation of Direct Behavior Ratings: A Multitrait Multimethod Analysis NASP Annual Convention 2014 Presenters: Dr. Faith Miller, NCSP Research Associate, University of Connecticut Daniel Cohen,
More information9 research designs likely for PSYC 2100
9 research designs likely for PSYC 2100 1) 1 factor, 2 levels, 1 group (one group gets both treatment levels) related samples t-test (compare means of 2 levels only) 2) 1 factor, 2 levels, 2 groups (one
More informationValidity and Reliability. PDF Created with deskpdf PDF Writer - Trial ::
Validity and Reliability PDF Created with deskpdf PDF Writer - Trial :: http://www.docudesk.com Validity Is the translation from concept to operationalization accurately representing the underlying concept.
More informationLecture 4: Research Approaches
Lecture 4: Research Approaches Lecture Objectives Theories in research Research design approaches ú Experimental vs. non-experimental ú Cross-sectional and longitudinal ú Descriptive approaches How to
More informationVARIABLES AND MEASUREMENT
ARTHUR SYC 204 (EXERIMENTAL SYCHOLOGY) 16A LECTURE NOTES [01/29/16] VARIABLES AND MEASUREMENT AGE 1 Topic #3 VARIABLES AND MEASUREMENT VARIABLES Some definitions of variables include the following: 1.
More informationCh. 11 Measurement. Measurement
TECH 646 Analysis of Research in Industry and Technology PART III The Sources and Collection of data: Measurement, Measurement Scales, Questionnaires & Instruments, Sampling Ch. 11 Measurement Lecture
More informationCh. 11 Measurement. Paul I-Hai Lin, Professor A Core Course for M.S. Technology Purdue University Fort Wayne Campus
TECH 646 Analysis of Research in Industry and Technology PART III The Sources and Collection of data: Measurement, Measurement Scales, Questionnaires & Instruments, Sampling Ch. 11 Measurement Lecture
More informationVALIDITY OF QUANTITATIVE RESEARCH
Validity 1 VALIDITY OF QUANTITATIVE RESEARCH Recall the basic aim of science is to explain natural phenomena. Such explanations are called theories (Kerlinger, 1986, p. 8). Theories have varying degrees
More informationValidity refers to the accuracy of a measure. A measurement is valid when it measures what it is suppose to measure and performs the functions that
Validity refers to the accuracy of a measure. A measurement is valid when it measures what it is suppose to measure and performs the functions that it purports to perform. Does an indicator accurately
More informationRevisiting the Halo Effect Within a Multitrait, Multimethod Framework
Revisiting the Halo Effect Within a Multitrait, Multimethod Framework Annual meeting of the American Educational Research Association San Francisco, CA Emily R. Lai Edward W. Wolfe Daisy H. Vickers April,
More information32.5. percent of U.S. manufacturers experiencing unfair currency manipulation in the trade practices of other countries.
TECH 646 Analysis of Research in Industry and Technology PART III The Sources and Collection of data: Measurement, Measurement Scales, Questionnaires & Instruments, Sampling Ch. 11 Measurement Lecture
More informationConstructing Indices and Scales. Hsueh-Sheng Wu CFDR Workshop Series June 8, 2015
Constructing Indices and Scales Hsueh-Sheng Wu CFDR Workshop Series June 8, 2015 1 Outline What are scales and indices? Graphical presentation of relations between items and constructs for scales and indices
More informationLecture Week 3 Quality of Measurement Instruments; Introduction SPSS
Lecture Week 3 Quality of Measurement Instruments; Introduction SPSS Introduction to Research Methods & Statistics 2013 2014 Hemmo Smit Overview Quality of Measurement Instruments Introduction SPSS Read:
More informationCHAPTER VI RESEARCH METHODOLOGY
CHAPTER VI RESEARCH METHODOLOGY 6.1 Research Design Research is an organized, systematic, data based, critical, objective, scientific inquiry or investigation into a specific problem, undertaken with the
More informationalternate-form reliability The degree to which two or more versions of the same test correlate with one another. In clinical studies in which a given function is going to be tested more than once over
More informationAssociate Prof. Dr Anne Yee. Dr Mahmoud Danaee
Associate Prof. Dr Anne Yee Dr Mahmoud Danaee 1 2 What does this resemble? Rorschach test At the end of the test, the tester says you need therapy or you can't work for this company 3 Psychological Testing
More informationResearch Questions and Survey Development
Research Questions and Survey Development R. Eric Heidel, PhD Associate Professor of Biostatistics Department of Surgery University of Tennessee Graduate School of Medicine Research Questions 1 Research
More informationSpecial Section QUANTITATIVE METHODS IN THE STUDY OF DRUG USE. Edited by George J. Huba ABSTRACT
J. DRUG EDUCATION, Vol. 14(2), 1984 Special Section QUANTITATIVE METHODS IN THE STUDY OF DRUG USE Edited by George J. Huba RELIABILITY OF SELF-REPORT MEASURES OF DRUG USE IN PREVENTION RESEARCH: Evaluation
More informationIntroduction to Reliability
Reliability Thought Questions: How does/will reliability affect what you do/will do in your future job? Which method of reliability analysis do you find most confusing? Introduction to Reliability What
More information26:010:557 / 26:620:557 Social Science Research Methods
26:010:557 / 26:620:557 Social Science Research Methods Dr. Peter R. Gillett Associate Professor Department of Accounting & Information Systems Rutgers Business School Newark & New Brunswick 1 Overview
More informationRESEARCH METHODS. Winfred, research methods, ; rv ; rv
RESEARCH METHODS 1 Research Methods means of discovering truth 2 Research Methods means of discovering truth what is truth? 3 Research Methods means of discovering truth what is truth? Riveda Sandhyavandanam
More information2 Types of psychological tests and their validity, precision and standards
2 Types of psychological tests and their validity, precision and standards Tests are usually classified in objective or projective, according to Pasquali (2008). In case of projective tests, a person is
More informationRESEARCH METHODS. Winfred, research methods,
RESEARCH METHODS Winfred, research methods, 04-23-10 1 Research Methods means of discovering truth Winfred, research methods, 04-23-10 2 Research Methods means of discovering truth what is truth? Winfred,
More informationHandout 5: Establishing the Validity of a Survey Instrument
In this handout, we will discuss different types of and methods for establishing validity. Recall that this concept was defined in Handout 3 as follows. Definition Validity This is the extent to which
More informationReliability & Validity Dr. Sudip Chaudhuri
Reliability & Validity Dr. Sudip Chaudhuri M. Sc., M. Tech., Ph.D., M. Ed. Assistant Professor, G.C.B.T. College, Habra, India, Honorary Researcher, Saha Institute of Nuclear Physics, Life Member, Indian
More informationMeasurement and Descriptive Statistics. Katie Rommel-Esham Education 604
Measurement and Descriptive Statistics Katie Rommel-Esham Education 604 Frequency Distributions Frequency table # grad courses taken f 3 or fewer 5 4-6 3 7-9 2 10 or more 4 Pictorial Representations Frequency
More informationChapter 4: Defining and Measuring Variables
Chapter 4: Defining and Measuring Variables A. LEARNING OUTCOMES. After studying this chapter students should be able to: Distinguish between qualitative and quantitative, discrete and continuous, and
More informationDATA is derived either through. Self-Report Observation Measurement
Data Management DATA is derived either through Self-Report Observation Measurement QUESTION ANSWER DATA DATA may be from Structured or Unstructured questions? Quantitative or Qualitative? Numerical or
More informationAP Psychology -- Chapter 02 Review Research Methods in Psychology
AP Psychology -- Chapter 02 Review Research Methods in Psychology 1. In the opening vignette, to what was Alicia's condition linked? The death of her parents and only brother 2. What did Pennebaker s study
More informationBasic concepts and principles of classical test theory
Basic concepts and principles of classical test theory Jan-Eric Gustafsson What is measurement? Assignment of numbers to aspects of individuals according to some rule. The aspect which is measured must
More informationValidity and Quantitative Research. What is Validity? What is Validity Cont. RCS /16/04
Validity and Quantitative Research RCS 6740 6/16/04 What is Validity? Valid Definition (Dictionary.com): Well grounded; just: a valid objection. Producing the desired results; efficacious: valid methods.
More informationPLS 506 Mark T. Imperial, Ph.D. Lecture Notes: Reliability & Validity
PLS 506 Mark T. Imperial, Ph.D. Lecture Notes: Reliability & Validity Measurement & Variables - Initial step is to conceptualize and clarify the concepts embedded in a hypothesis or research question with
More informationMeasurement. Different terminology. Marketing managers work with abstractions. Different terminology. Different terminology.
Different terminology Managerial view Marketing managers work with abstractions Concepts A generalized idea about a class of objects, attributes, occurrences or processes Propositions A set of concepts
More informationEvaluating Reading Diagnostic Tests:
Evaluating Reading Diagnostic Tests: An Application of Confirmatory Factor Analysis To Multitrait-Multimethod Data Herbert W. Marsh and Susan Butler The University of Sydney Diagnostic reading tests, in
More informationMeasurement. 500 Research Methods Mike Kroelinger
Measurement 500 Research Methods Mike Kroelinger Levels of Measurement Nominal Lowest level -- used to classify variables into two or more categories. Cases placed in the same category must be equivalent.
More informationSTATISTICAL CONCLUSION VALIDITY
Validity 1 The attached checklist can help when one is evaluating the threats to validity of a study. VALIDITY CHECKLIST Recall that these types are only illustrative. There are many more. INTERNAL VALIDITY
More informationHPS301 Exam Notes- Contents
HPS301 Exam Notes- Contents Week 1 Research Design: What characterises different approaches 1 Experimental Design 1 Key Features 1 Criteria for establishing causality 2 Validity Internal Validity 2 Threats
More informationExamining the efficacy of the Theory of Planned Behavior (TPB) to understand pre-service teachers intention to use technology*
Examining the efficacy of the Theory of Planned Behavior (TPB) to understand pre-service teachers intention to use technology* Timothy Teo & Chwee Beng Lee Nanyang Technology University Singapore This
More informationVariable Measurement, Norms & Differences
Variable Measurement, Norms & Differences 1 Expectations Begins with hypothesis (general concept) or question Create specific, testable prediction Prediction can specify relation or group differences Different
More informationReliability and Validity
Reliability and Today s Objectives Understand the difference between reliability and validity Understand how to develop valid indicators of a concept Reliability and Reliability How accurate or consistent
More informationOn the purpose of testing:
Why Evaluation & Assessment is Important Feedback to students Feedback to teachers Information to parents Information for selection and certification Information for accountability Incentives to increase
More informationValidity, Reliability and Classical Assumptions
, Reliability and Classical Assumptions Presented by Mahendra AN Sources: www-psych.stanford.edu/~bigopp/.ppt http://ets.mnsu.edu/darbok/ethn402-502/reliability.ppt http://5martconsultingbandung.blogspot.com/2011/01/uji-asumsi-klasik.html
More information9.63 Laboratory in Cognitive Science
9.63 Laboratory in Cognitive Science Fall 2005 Course 2b Variables, Controls Aude Oliva Ben Balas, Charles Kemp Science is: Scientific Thinking 1. Empirical Based on observations 2. Objective Observations
More informationVariables and Data. Gbenga Ogunfowokan Lead, Nigerian Regional Faculty The Global Health Network 19 th May 2017
Variables and Data Gbenga Ogunfowokan Lead, Nigerian Regional The Global Health Network 19 th May 2017 Objectives At the end of this presentation you should be able to 1) Define a variable 2) Classify
More informationVariables in Research. What We Will Cover in This Section. What Does Variable Mean?
Variables in Research 9/20/2005 P767 Variables in Research 1 What We Will Cover in This Section Nature of variables. Measuring variables. Reliability. Validity. Measurement Modes. Issues. 9/20/2005 P767
More informationUsing Analytical and Psychometric Tools in Medium- and High-Stakes Environments
Using Analytical and Psychometric Tools in Medium- and High-Stakes Environments Greg Pope, Analytics and Psychometrics Manager 2008 Users Conference San Antonio Introduction and purpose of this session
More informationDesigning a Questionnaire
Designing a Questionnaire What Makes a Good Questionnaire? As a rule of thumb, never to attempt to design a questionnaire! A questionnaire is very easy to design, but a good questionnaire is virtually
More informationEmpowered by Psychometrics The Fundamentals of Psychometrics. Jim Wollack University of Wisconsin Madison
Empowered by Psychometrics The Fundamentals of Psychometrics Jim Wollack University of Wisconsin Madison Psycho-what? Psychometrics is the field of study concerned with the measurement of mental and psychological
More informationPsy 464 Advanced Experimental Design. Overview. Variables types. Variables, Control, Validity & Levels of Measurement
Psy 464 Advanced Experimental Design Variables, Control, Validity & Levels of Measurement Overview Variables and operationalization Control and confounding Validity Levels of measurement SPSS introduction/overview
More informationVariables in Research. What We Will Cover in This Section. What Does Variable Mean? Any object or event that can take on more than one form or value.
Variables in Research 1/1/2003 P365 Variables in Research 1 What We Will Cover in This Section Nature of variables. Measuring variables. Reliability. Validity. Measurement Modes. Issues. 1/1/2003 P365
More informationDATA GATHERING. Define : Is a process of collecting data from sample, so as for testing & analyzing before reporting research findings.
DATA GATHERING Define : Is a process of collecting data from sample, so as for testing & analyzing before reporting research findings. 2012 John Wiley & Sons Ltd. Measurement Measurement: the assignment
More informationAnalysis of Environmental Data Conceptual Foundations: En viro n m e n tal Data
Analysis of Environmental Data Conceptual Foundations: En viro n m e n tal Data 1. Purpose of data collection...................................................... 2 2. Samples and populations.......................................................
More information04/12/2014. Research Methods in Psychology. Chapter 6: Independent Groups Designs. What is your ideas? Testing
Research Methods in Psychology Chapter 6: Independent Groups Designs 1 Why Psychologists Conduct Experiments? What is your ideas? 2 Why Psychologists Conduct Experiments? Testing Hypotheses derived from
More informationUNIVERSITY OF THE FREE STATE DEPARTMENT OF COMPUTER SCIENCE AND INFORMATICS CSIS6813 MODULE TEST 2
UNIVERSITY OF THE FREE STATE DEPARTMENT OF COMPUTER SCIENCE AND INFORMATICS CSIS6813 MODULE TEST 2 DATE: 3 May 2017 MARKS: 75 ASSESSOR: Prof PJ Blignaut MODERATOR: Prof C de Villiers (UP) TIME: 2 hours
More informationValidation of Scales
Validation of Scales ἀγεωμέτρητος μηδεὶς εἰσίτω (Let none enter without a knowledge of mathematics) D R D I N E S H R A M O O Introduction Validity and validation are crucial to understanding psychological
More informationGroup Assignment #1: Concept Explication. For each concept, ask and answer the questions before your literature search.
Group Assignment #1: Concept Explication 1. Preliminary identification of the concept. Identify and name each concept your group is interested in examining. Questions to asked and answered: Is each concept
More informationReliability AND Validity. Fact checking your instrument
Reliability AND Validity Fact checking your instrument General Principles Clearly Identify the Construct of Interest Use Multiple Items Use One or More Reverse Scored Items Use a Consistent Response Format
More informationINTERPARENTAL CONFLICT, ADOLESCENT BEHAVIORAL PROBLEMS, AND ADOLESCENT COMPETENCE: CONVERGENT AND DISCRIMINANT VALIDITY
EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 10.1177/0013164403258462 EPSTEIN ET AL. ARTICLE INTERPARENTAL CONFLICT, ADOLESCENT BEHAVIORAL PROBLEMS, AND ADOLESCENT COMPETENCE: CONVERGENT AND DISCRIMINANT
More information11/18/2013. Correlational Research. Correlational Designs. Why Use a Correlational Design? CORRELATIONAL RESEARCH STUDIES
Correlational Research Correlational Designs Correlational research is used to describe the relationship between two or more naturally occurring variables. Is age related to political conservativism? Are
More informationUse of the Quantitative-Methods Approach in Scientific Inquiry. Du Feng, Ph.D. Professor School of Nursing University of Nevada, Las Vegas
Use of the Quantitative-Methods Approach in Scientific Inquiry Du Feng, Ph.D. Professor School of Nursing University of Nevada, Las Vegas The Scientific Approach to Knowledge Two Criteria of the Scientific
More informationWork, Employment, and Industrial Relations Theory Spring 2008
MIT OpenCourseWare http://ocw.mit.edu 15.676 Work, Employment, and Industrial Relations Theory Spring 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
More informationRELIABILITY AND VALIDITY (EXTERNAL AND INTERNAL)
UNIT 2 RELIABILITY AND VALIDITY (EXTERNAL AND INTERNAL) Basic Process/Concept in Research Structure 2.0 Introduction 2.1 Objectives 2.2 Reliability 2.3 Methods of Estimating Reliability 2.3.1 External
More informationRegression Discontinuity Analysis
Regression Discontinuity Analysis A researcher wants to determine whether tutoring underachieving middle school students improves their math grades. Another wonders whether providing financial aid to low-income
More informationOn the usefulness of the CEFR in the investigation of test versions content equivalence HULEŠOVÁ, MARTINA
On the usefulness of the CEFR in the investigation of test versions content equivalence HULEŠOVÁ, MARTINA MASARY K UNIVERSITY, CZECH REPUBLIC Overview Background and research aims Focus on RQ2 Introduction
More informationDo not write your name on this examination all 40 best
Student #: Do not write your name on this examination Research in Psychology I; Final Exam Fall 200 Instructor: Jeff Aspelmeier NOTE: THIS EXAMINATION MAY NOT BE RETAINED BY STUDENTS This exam is worth
More informationINTRODUCTION TO STATISTICS SORANA D. BOLBOACĂ
INTRODUCTION TO STATISTICS SORANA D. BOLBOACĂ OBJECTIVES Definitions Stages of Scientific Knowledge Quantification and Accuracy Types of Medical Data Population and sample Sampling methods DEFINITIONS
More informationTechnical Whitepaper
Technical Whitepaper July, 2001 Prorating Scale Scores Consequential analysis using scales from: BDI (Beck Depression Inventory) NAS (Novaco Anger Scales) STAXI (State-Trait Anxiety Inventory) PIP (Psychotic
More informationBusiness Statistics Probability
Business Statistics The following was provided by Dr. Suzanne Delaney, and is a comprehensive review of Business Statistics. The workshop instructor will provide relevant examples during the Skills Assessment
More informationQuantifying Construct Validity: Two Simple Measures
Quantifying Construct Validity: Two Simple Measures The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters. Citation Published Version
More informationDoctoral Dissertation Boot Camp Quantitative Methods Kamiar Kouzekanani, PhD January 27, The Scientific Method of Problem Solving
Doctoral Dissertation Boot Camp Quantitative Methods Kamiar Kouzekanani, PhD January 27, 2018 The Scientific Method of Problem Solving The conceptual phase Reviewing the literature, stating the problem,
More informationOptions and issues in the development of validation methodologies for Hua Oranga and Hōmai te Waiora ki Ahau
Options and issues in the development of validation methodologies for Hua Oranga and Hōmai te Waiora ki Ahau The following paper sets out to describe options and issues which need to be considered in the
More informationPsychological testing
Psychological testing Lecture 12 Mikołaj Winiewski, PhD Test Construction Strategies Content validation Empirical Criterion Factor Analysis Mixed approach (all of the above) Content Validation Defining
More informationESTABLISHING VALIDITY AND RELIABILITY OF ACHIEVEMENT TEST IN BIOLOGY FOR STD. IX STUDENTS
International Journal of Educational Science and Research (IJESR) ISSN(P): 2249-6947; ISSN(E): 2249-8052 Vol. 4, Issue 4, Aug 2014, 29-36 TJPRC Pvt. Ltd. ESTABLISHING VALIDITY AND RELIABILITY OF ACHIEVEMENT
More informationEmpirical Knowledge: based on observations. Answer questions why, whom, how, and when.
INTRO TO RESEARCH METHODS: Empirical Knowledge: based on observations. Answer questions why, whom, how, and when. Experimental research: treatments are given for the purpose of research. Experimental group
More informationReliability. Internal Reliability
32 Reliability T he reliability of assessments like the DECA-I/T is defined as, the consistency of scores obtained by the same person when reexamined with the same test on different occasions, or with
More information11/24/2017. Do not imply a cause-and-effect relationship
Correlational research is used to describe the relationship between two or more naturally occurring variables. Is age related to political conservativism? Are highly extraverted people less afraid of rejection
More informationMeasurement issues in the assessment of psychosocial stressors at work
Measurement issues in the assessment of psychosocial stressors at work Dr. Peter Schnall And Dr. Paul Landsbergis Session 3 2 nd Hour April 18 2012 Reliability* The degree of stability exhibited when a
More informationResults & Statistics: Description and Correlation. I. Scales of Measurement A Review
Results & Statistics: Description and Correlation The description and presentation of results involves a number of topics. These include scales of measurement, descriptive statistics used to summarize
More informationChapter 11 Nonexperimental Quantitative Research Steps in Nonexperimental Research
Chapter 11 Nonexperimental Quantitative Research (Reminder: Don t forget to utilize the concept maps and study questions as you study this and the other chapters.) Nonexperimental research is needed because
More informationOverview of Experimentation
The Basics of Experimentation Overview of Experiments. IVs & DVs. Operational Definitions. Reliability. Validity. Internal vs. External Validity. Classic Threats to Internal Validity. Lab: FP Overview;
More informationSaville Consulting Wave Professional Styles Handbook
Saville Consulting Wave Professional Styles Handbook PART 4: TECHNICAL Chapter 19: Reliability This manual has been generated electronically. Saville Consulting do not guarantee that it has not been changed
More informationUnderlying Theory & Basic Issues
Underlying Theory & Basic Issues Dewayne E Perry ENS 623 Perry@ece.utexas.edu 1 All Too True 2 Validity In software engineering, we worry about various issues: E-Type systems: Usefulness is it doing what
More informationChoosing an Approach for a Quantitative Dissertation: Strategies for Various Variable Types
Choosing an Approach for a Quantitative Dissertation: Strategies for Various Variable Types Kuba Glazek, Ph.D. Methodology Expert National Center for Academic and Dissertation Excellence Outline Thesis
More information(CORRELATIONAL DESIGN AND COMPARATIVE DESIGN)
UNIT 4 OTHER DESIGNS (CORRELATIONAL DESIGN AND COMPARATIVE DESIGN) Quasi Experimental Design Structure 4.0 Introduction 4.1 Objectives 4.2 Definition of Correlational Research Design 4.3 Types of Correlational
More informationDescribe what is meant by a placebo Contrast the double-blind procedure with the single-blind procedure Review the structure for organizing a memo
Please note the page numbers listed for the Lind book may vary by a page or two depending on which version of the textbook you have. Readings: Lind 1 11 (with emphasis on chapters 5, 6, 7, 8, 9 10 & 11)
More informationUrsuline College Accelerated Program
Ursuline College Accelerated Program CRITICAL INFORMATION! DO NOT SKIP THIS LINK BELOW... BEFORE PROCEEDING TO READ THE UCAP MODULE, YOU ARE EXPECTED TO READ AND ADHERE TO ALL UCAP POLICY INFORMATION CONTAINED
More informationBuy full version here - for $ 7.00
This is a Sample version of the Apathy Evaluation Scale (AES) The full version of watermark.. the AES comes without sample The full complete 40 page version includes AES Overview information AES Scoring/
More informationsample of 85 graduate students at The University of Michigan s influenza, benefits provided by a flu shot, and the barriers or costs associated
Construct Validation of the Health Belief Model K. Michael Cummings, M.P.H. Alan M. Jette, M.P.H. Irwin M. Rosenstock, Ph.D.* A multitrait-multimethod design was employed to assess the construct validity
More informationOn the Many Claims and Applications of the Latent Variable
On the Many Claims and Applications of the Latent Variable Science is an attempt to exploit this contact between our minds and the world, and science is also motivated by the limitations that result from
More informationUsing the Rasch Modeling for psychometrics examination of food security and acculturation surveys
Using the Rasch Modeling for psychometrics examination of food security and acculturation surveys Jill F. Kilanowski, PhD, APRN,CPNP Associate Professor Alpha Zeta & Mu Chi Acknowledgements Dr. Li Lin,
More informationBy Hui Bian Office for Faculty Excellence
By Hui Bian Office for Faculty Excellence 1 Email: bianh@ecu.edu Phone: 328-5428 Location: 1001 Joyner Library, room 1006 Office hours: 8:00am-5:00pm, Monday-Friday 2 Educational tests and regular surveys
More informationUsing Generalizability Theory to Investigate the Psychometric Property of an Assessment Center in Indonesia
Using Generalizability Theory to Investigate the Psychometric Property of an Assessment Center in Indonesia Urip Purwono received his Ph.D. (psychology) from the University of Massachusetts at Amherst,
More informationMULTIPLE CHOICE. Choose the one alternative that best completes the statement or answers the question.
381 Exam #2 E Name MULTIPLE CHOICE. Choose the one alternative that best completes the statement or answers the question. 1) An approach to qualitative research that focuses on building theories is known
More informationThe Current State of Our Education
1 The Current State of Our Education 2 Quantitative Research School of Management www.ramayah.com Mental Challenge A man and his son are involved in an automobile accident. The man is killed and the boy,
More informationDescribe what is meant by a placebo Contrast the double-blind procedure with the single-blind procedure Review the structure for organizing a memo
Please note the page numbers listed for the Lind book may vary by a page or two depending on which version of the textbook you have. Readings: Lind 1 11 (with emphasis on chapters 10, 11) Please note chapter
More informationReliability of the Ghiselli Self-Description Inventory as a Measure of Self-Esteem
An Examination of the Construct Validity and Reliability of the Ghiselli Self-Description Inventory as a Measure of Self-Esteem Charles S. Raben Personnel Research and Development Center, U.S. Civil Service
More information