Making a psychometric. Dr Benjamin Cowan- Lecture 9

Size: px
Start display at page:

Download "Making a psychometric. Dr Benjamin Cowan- Lecture 9"

Transcription

1 Making a psychometric Dr Benjamin Cowan- Lecture 9

2 What this lecture will cover What is a questionnaire? Development of questionnaires Item development Scale options Scale reliability & validity Factor Analysis

3 What is a questionnaire? Some concepts are difficult to measure directly using measurements like time, accuracy etc Attitudes, emotions, opinions We need to design psychometrics for these if we are to research them

4 Why would we want to make a psychometric? If we are looking at a new concept that hasn t been measured before Happens a lot in HCI with developments of new technologies Because a metric needs to measure something specific for it to have value, we need to design or tweak existing measures for new technologies Need to add items and re-test

5 Example- Anxiety towards facebook posting Let s say we wanted to make a measure of how anxious people were about posting to facebook This measure (our questionnaire) is made of attitude phrases (or items).

6 Stages of item development Literature review What are the key concepts in studying anxiety? Measure review What is available? How is anxiety currently measured? Focus groups/interviews What is important in facebook anxiety? Questions about facebook and negative emotions Gives an indication of how people describe the concepts, thus improving item wording

7 Generating Items- Interviews Conversation with a purpose 4 main types Unstructured Semi Structured Structured Group

8 Unstructured interviews Exploratory Talk around an area Planning the areas for discussion rather than specific questions Can explore topics as they come up

9 Structured Interviews Predetermined questions Standardised for all interviewees

10 Semi Structured Interviews Basic script used with all participants Mix of Structured and Unstructured Interview There are some questions that are covered with all and the rest is a free flowing conversation

11 What interview type to use? Depends on: How specific you need to get Purpose of the interview

12 Stages of item development This will allow you to get an idea of: Potential items Potential categories that need to be covered (factors) Pilot study Large number of items Participants rate: Clarity of wording Clarity of concept in the item Experts in the area to review items

13 The good, the bad, the ugly Good item Clear, well worded, one concept, to the point. I feel stressed when using facebook Bad item Can be clearly worded but does not cover one concept I feel stressed because of so many people on facebook and it is hard to use Ugly item Poorly worded and doesn t cover one concept Stress is something I feel all of the time when using facebook because people on it are plentiful and it s difficult This can happen when questionnaires are mis-translated.

14 Common scales used Likert Scales (Likert, 1926) 3 point, 5 point, 7 point, 9 point More points, the larger the variance of responses on item Arguments over which is best but 5 point is most common The use of a neutral point is also debated Semantic Differential Uses two polar opposite adjectives at the end of a scale Which to use? Strong-Not Strong (bad) Strong- Weak (good)

15 Important concepts in item response Response Acquiescence set A propensity for participants to answer positively to items Balancing psychometric as much as possible (positively and negatively worded items) Item Randomisation Social Desirability Responding with what you feel is socially appropriate

16 So We have our items We have piloted them with participants We now need to assess how good our questionnaire (or psychometric) is Good psychometrics have: High reliability High validity Possess a set of norms (baselines/guides)

17 Reliability Stability of the test score over time Test-Retest Reliability Internal consistency of the test Internal consistency reliability The extent to which the items are measuring the same underlying concept

18 Test-Retest Reliability Test at Time 1 6 month gap Test at Time 2 Testing same participants on the measure on two occasions Scores are then correlated to see strength of relationship Over 0.7 is good test- retest reliability

19 Why would the correlation not be perfect? Between times there may be changes on the variables Some people may have become less anxious over time Test Error N feeling ill, bored, tired.

20 Internal consistency reliability The extent to which each item measures the same underlying concept In our facebook posting anxiety scale we would expect all the items to be measuring elements of anxiety not measuring usability of facebook

21 Internal consistency measures Split Half method Divide measure in two randomly and correlate the scores on the two halves together Cronbach alpha (most commonly used) Average correlation of all possible split half correlations. 0.7 seen as a good alpha

22 What can impact on this reliability The number of items More items mean more of concept can be covered Weighing up number of items and boredom 10 items considered minimum for reliable test Can a measure be too internally consistent? (Cattell, 1957) Using items which effectively measure the same thing E.g. I like facebook and Facebook is something I like They are the same item, just different wording Leads to a bloated specific

23 Cronbach alpha analysis The analysis looks at all correlations of the item scores with the total questionnaire score (itemtotal correlations) Items with Item-total correlations of lower than 0.3 should be removed as they do not correlate well The test output also gives us an idea of what alpha would be without each item- great for item removal

24 Validity of a test A test can be reliable but not valid It could be high in reliability but not measuring what it proclaims to measure It is not as simple as looking at the item wordings to deduce this We need to identify whether our measure behaves as predicted

25 Validity Assessment Face validity The items seem to be worded right for the concept being measured This is a poor test of validity E.g. I am quite easily distracted - looks fine but can be interpreted differently by participants Concurrent Validity Correlation of test with other benchmark test that was given at the same time Dubious when there is no clear benchmark

26 Validity Assessment Predictive Validity The measure is able to predict some criterion E.g. facebook anxiety relates to posting behaviour Need to be aware that modest relationships are likely Many other factors important to posting behaviour closeness of facebook friends, drunken messaging? Sometimes clear criterions are not available Beware of the difference between statistical significance and psychological significance

27 Construct Validity (Cronbach & Meehl, 1955) Allows a collection of results to lead us to validity conclusions rather than just one Usually the case that not all hypotheses are confirmed Validity is therefore not as equivocal as reliability Interpretive and subjective

28 Construct Validity (Cronbach & Meehl, 1955) Construct Validity A bank of hypotheses based on the knowledge of our concept Our Hypotheses for Facebook anxiety Should correlate positively and highly with other measures of anxiety (concurrent validity) Should correlate positively with someone s fear of negative evaluation (concurrent validity) Should not correlate with personality tests that don t measure anxiety High scorers, compared with low scorers should show less activity on facebook, and more leaving facebook (predictive validity)

29 Norms We need to test our measures on A significant representative proportion of the population (1000 s of respondents) A sample of people we d expect to be high or low on the measure (for discriminatory markers) This is built up over years of use

30 Now we have Gathered our items Assessed their reliability Assessed their validity We are assuming at present that facebook anxiety is uni-dimensional. This might not be true, there may be many factors to it, which we have picked up in our measure

31 What are factors? Each questionnaire item gives a score There will items that correlate heavily together Factor analysis is fundamentally used to: reduce the data into the smallest number of explanatory concepts A factor is a combination of variables, the grouping of which indicates a relationship

32 What are factors? Each item has a factor loading correlation of that item with the factor Some items will have high loadings, some low or no loading at all on a specific factor Loadings of 0.4 are seen as helpful in defining a factor Items should only load heavily on one factor If they don t they are candidates for rewording

33 Shared Variance Correlation co-efficient represents The amount of agreement (or shared variance) between two sets of scores Square the correlation coefficient to get % agreement Variable x variance Shared (Common) Variance Variable y variance

34 Shared Variance & Communality By squaring the factor loading we can: Identify how much shared variance there is between the item and the factor They can be thought of as the contribution that the item makes to the factor If we do this for each factor loading an item has we get the item s communality the amount of variance shared between the item and all the factors

35 Factor Extraction Eigenvalues Indicate the importance of the factor extracted in explaining the variance in the data There will be few with high eigenvalues and lots with low Makes sense to keep the most important factors Rule of thumb is keep factors with eigenvalues > 1 (as an eigenvalue of 1 represents a significant amount of variation). The number to extract is identified using a Scree Plot (Cattell, 1966) Y axis is eigenvalues X axis is the number of factors

36 Scree Plot Eigenvalues Point of Inflexion Number of Factors

37 Factor Rotation Looking for best fit - factor structure with clearest interpretation Sometimes this involves rotation to get the clearest, simplest factor structure A simple factor structure is one that has a few high loading items and the rest being near 0 (Cattell, 1978)

38 Methods of Rotation The method you choose depends on how correlated you feel the factor scores should be Based on theoretical reasoning We would expect our questionnaire To have factors- 1) anxiety about social posting, 2) anxiety about interface interaction, 3) social confidence For the scores from this to be correlated

39 Methods of Rotation We would therefore use a method that takes this correlation into consideration- Direct Oblimin This is an oblique method of rotation (allows the factors to correlate)

40 Methods of Rotation If we felt they should not correlated then we could have used Varimax method This is an example of orthogonal rotation- ensures the extracted factors are not correlated.

41 Considerations Sample size Number of people in the sample debated 100 for stable factors (Kline, 1999)

42 Using Factor Analysis in questionnaire construction Give participants questionnaire Conduct factor analysis Any that load highly on more than one factor, check for concept clarity Check that all those with loadings >0.3 cover the most of what we need in the scale, if not write more items Replicate this on each new sample Validate the scale factors and calculate their reliability

43 Making a psychometric Takes a lot of time To develop the items To test on wide range of samples To test a large bank of hypotheses on relationships to ensure its validity Sometimes it cannot be avoided

44 Readings Kline, P. (2000). A Psychometrics Primer, Chapter 3. Free Association Books from Amazon Kline (1994). An easy guide to factor analysis (available in library) Field, A. (2007).Chapter 15- Exploratory Factor Analysis

Reliability and Validity checks S-005

Reliability and Validity checks S-005 Reliability and Validity checks S-005 Checking on reliability of the data we collect Compare over time (test-retest) Item analysis Internal consistency Inter-rater agreement Compare over time Test-Retest

More information

Summary & Conclusion. Lecture 10 Survey Research & Design in Psychology James Neill, 2016 Creative Commons Attribution 4.0

Summary & Conclusion. Lecture 10 Survey Research & Design in Psychology James Neill, 2016 Creative Commons Attribution 4.0 Summary & Conclusion Lecture 10 Survey Research & Design in Psychology James Neill, 2016 Creative Commons Attribution 4.0 Overview 1. Survey research and design 1. Survey research 2. Survey design 2. Univariate

More information

By Hui Bian Office for Faculty Excellence

By Hui Bian Office for Faculty Excellence By Hui Bian Office for Faculty Excellence 1 Email: bianh@ecu.edu Phone: 328-5428 Location: 1001 Joyner Library, room 1006 Office hours: 8:00am-5:00pm, Monday-Friday 2 Educational tests and regular surveys

More information

Survey research (Lecture 1) Summary & Conclusion. Lecture 10 Survey Research & Design in Psychology James Neill, 2015 Creative Commons Attribution 4.

Survey research (Lecture 1) Summary & Conclusion. Lecture 10 Survey Research & Design in Psychology James Neill, 2015 Creative Commons Attribution 4. Summary & Conclusion Lecture 10 Survey Research & Design in Psychology James Neill, 2015 Creative Commons Attribution 4.0 Overview 1. Survey research 2. Survey design 3. Descriptives & graphing 4. Correlation

More information

Survey research (Lecture 1)

Survey research (Lecture 1) Summary & Conclusion Lecture 10 Survey Research & Design in Psychology James Neill, 2015 Creative Commons Attribution 4.0 Overview 1. Survey research 2. Survey design 3. Descriptives & graphing 4. Correlation

More information

Unit outcomes. Summary & Conclusion. Lecture 10 Survey Research & Design in Psychology James Neill, 2018 Creative Commons Attribution 4.0.

Unit outcomes. Summary & Conclusion. Lecture 10 Survey Research & Design in Psychology James Neill, 2018 Creative Commons Attribution 4.0. Summary & Conclusion Image source: http://commons.wikimedia.org/wiki/file:pair_of_merops_apiaster_feeding_cropped.jpg Lecture 10 Survey Research & Design in Psychology James Neill, 2018 Creative Commons

More information

Unit outcomes. Summary & Conclusion. Lecture 10 Survey Research & Design in Psychology James Neill, 2018 Creative Commons Attribution 4.0.

Unit outcomes. Summary & Conclusion. Lecture 10 Survey Research & Design in Psychology James Neill, 2018 Creative Commons Attribution 4.0. Summary & Conclusion Image source: http://commons.wikimedia.org/wiki/file:pair_of_merops_apiaster_feeding_cropped.jpg Lecture 10 Survey Research & Design in Psychology James Neill, 2018 Creative Commons

More information

Problem Situation Form for Parents

Problem Situation Form for Parents Problem Situation Form for Parents Please complete a form for each situation you notice causes your child social anxiety. 1. WHAT WAS THE SITUATION? Please describe what happened. Provide enough information

More information

Worries and Anxiety F O R K I D S. C o u n s e l l i n g D i r e c t o r y H a p p i f u l K i d s

Worries and Anxiety F O R K I D S. C o u n s e l l i n g D i r e c t o r y H a p p i f u l K i d s Worries and Anxiety F O R K I D S C o u n s e l l i n g D i r e c t o r y H a p p i f u l K i d s Contents What is anxiety? Types of anxiety What does anxiety look like? Top tips for tackling worries Asking

More information

Designing a Questionnaire

Designing a Questionnaire Designing a Questionnaire What Makes a Good Questionnaire? As a rule of thumb, never to attempt to design a questionnaire! A questionnaire is very easy to design, but a good questionnaire is virtually

More information

Survey Research. We can learn a lot simply by asking people what we want to know... THE PREVALENCE OF SURVEYS IN COMMUNICATION RESEARCH

Survey Research. We can learn a lot simply by asking people what we want to know... THE PREVALENCE OF SURVEYS IN COMMUNICATION RESEARCH Survey Research From surveys we can learn how large groups of people think and act. To trust generalizations made on the basis of surveys, however, the sample must be representative, the response rate

More information

ADMS Sampling Technique and Survey Studies

ADMS Sampling Technique and Survey Studies Principles of Measurement Measurement As a way of understanding, evaluating, and differentiating characteristics Provides a mechanism to achieve precision in this understanding, the extent or quality As

More information

Product Interest and Engagement Scale, Beta (PIES-beta): Initial Development

Product Interest and Engagement Scale, Beta (PIES-beta): Initial Development Product Interest and Engagement Scale, Beta (PIES-beta): Initial Development Christopher N. Chapman Microsoft Corporation Microsoft Way (cchap) Redmond, WA 98052 USA chris.chapman@microsoft.com Michal

More information

CHAPTER III RESEARCH METHODOLOGY

CHAPTER III RESEARCH METHODOLOGY CHAPTER III RESEARCH METHODOLOGY Research methodology explains the activity of research that pursuit, how it progress, estimate process and represents the success. The methodological decision covers the

More information

alternate-form reliability The degree to which two or more versions of the same test correlate with one another. In clinical studies in which a given function is going to be tested more than once over

More information

Internal structure evidence of validity

Internal structure evidence of validity Internal structure evidence of validity Dr Wan Nor Arifin Lecturer, Unit of Biostatistics and Research Methodology, Universiti Sains Malaysia. E-mail: wnarifin@usm.my Wan Nor Arifin, 2017. Internal structure

More information

Research Methods 1 Handouts, Graham Hole,COGS - version 1.0, September 2000: Page 1:

Research Methods 1 Handouts, Graham Hole,COGS - version 1.0, September 2000: Page 1: Research Methods 1 Handouts, Graham Hole,COGS - version 10, September 000: Page 1: T-TESTS: When to use a t-test: The simplest experimental design is to have two conditions: an "experimental" condition

More information

Psychometric Properties of the Mean Opinion Scale

Psychometric Properties of the Mean Opinion Scale Psychometric Properties of the Mean Opinion Scale James R. Lewis IBM Voice Systems 1555 Palm Beach Lakes Blvd. West Palm Beach, Florida jimlewis@us.ibm.com Abstract The Mean Opinion Scale (MOS) is a seven-item

More information

Procedia - Social and Behavioral Sciences 205 ( 2015 ) th World conference on Psychology Counseling and Guidance, May 2015

Procedia - Social and Behavioral Sciences 205 ( 2015 ) th World conference on Psychology Counseling and Guidance, May 2015 Available online at www.sciencedirect.com ScienceDirect Procedia - Social and Behavioral Sciences 205 ( 2015 ) 480 484 6th World conference on Psychology Counseling and Guidance, 14-16 May 2015 Reasons

More information

FACTOR ANALYSIS Factor Analysis 2006

FACTOR ANALYSIS Factor Analysis 2006 FACTOR ANALYSIS Factor Analysis 2006 Lecturer: Timothy Bates Tim.bates@ed.ac.uk Lecture Notes based on Austin 2005 Bring your hand out to the tutorial Please read prior to the tutorial A statistical tool

More information

Psychometric Instrument Development

Psychometric Instrument Development Psychometric Instrument Development Image source: http://commons.wikimedia.org/wiki/file:soft_ruler.jpg, CC-by-SA 3.0 Lecture 6 Survey Research & Design in Psychology James Neill, 2017 Creative Commons

More information

Psychometric Instrument Development

Psychometric Instrument Development Psychometric Instrument Development Image source: http://commons.wikimedia.org/wiki/file:soft_ruler.jpg, CC-by-SA 3.0 Lecture 6 Survey Research & Design in Psychology James Neill, 2017 Creative Commons

More information

Examining the Psychometric Properties of The McQuaig Occupational Test

Examining the Psychometric Properties of The McQuaig Occupational Test Examining the Psychometric Properties of The McQuaig Occupational Test Prepared for: The McQuaig Institute of Executive Development Ltd., Toronto, Canada Prepared by: Henryk Krajewski, Ph.D., Senior Consultant,

More information

Lecture Week 3 Quality of Measurement Instruments; Introduction SPSS

Lecture Week 3 Quality of Measurement Instruments; Introduction SPSS Lecture Week 3 Quality of Measurement Instruments; Introduction SPSS Introduction to Research Methods & Statistics 2013 2014 Hemmo Smit Overview Quality of Measurement Instruments Introduction SPSS Read:

More information

Psychometric Instrument Development

Psychometric Instrument Development Psychometric Instrument Development Lecture 6 Survey Research & Design in Psychology James Neill, 2012 Readings: Psychometrics 1. Bryman & Cramer (1997). Concepts and their measurement. [chapter - ereserve]

More information

Higher Psychology RESEARCH REVISION

Higher Psychology RESEARCH REVISION Higher Psychology RESEARCH REVISION 1 The biggest change from the old Higher course (up to 2014) is the possibility of an analysis and evaluation question (8-10) marks asking you to comment on aspects

More information

Dimensionality, internal consistency and interrater reliability of clinical performance ratings

Dimensionality, internal consistency and interrater reliability of clinical performance ratings Medical Education 1987, 21, 130-137 Dimensionality, internal consistency and interrater reliability of clinical performance ratings B. R. MAXIMt & T. E. DIELMANS tdepartment of Mathematics and Statistics,

More information

Measurement and Descriptive Statistics. Katie Rommel-Esham Education 604

Measurement and Descriptive Statistics. Katie Rommel-Esham Education 604 Measurement and Descriptive Statistics Katie Rommel-Esham Education 604 Frequency Distributions Frequency table # grad courses taken f 3 or fewer 5 4-6 3 7-9 2 10 or more 4 Pictorial Representations Frequency

More information

Importance of Good Measurement

Importance of Good Measurement Importance of Good Measurement Technical Adequacy of Assessments: Validity and Reliability Dr. K. A. Korb University of Jos The conclusions in a study are only as good as the data that is collected. The

More information

Psychometric Instrument Development

Psychometric Instrument Development Psychometric Instrument Development Image source: http://commons.wikimedia.org/wiki/file:soft_ruler.jpg, CC-by-SA 3.0 Lecture 6 Survey Research & Design in Psychology James Neill, 2016 Creative Commons

More information

Comparing 3 Means- ANOVA

Comparing 3 Means- ANOVA Comparing 3 Means- ANOVA Evaluation Methods & Statistics- Lecture 7 Dr Benjamin Cowan Research Example- Theory of Planned Behaviour Ajzen & Fishbein (1981) One of the most prominent models of behaviour

More information

Cognitive Restructuring

Cognitive Restructuring Cognitive Restructuring Cognitive Restructuring Cognitive Restructuring is an evidence based intervention for the treatment of low mood or anxiety, recommended by the National Institute for Health and

More information

CHAPTER 3 METHOD AND PROCEDURE

CHAPTER 3 METHOD AND PROCEDURE CHAPTER 3 METHOD AND PROCEDURE Previous chapter namely Review of the Literature was concerned with the review of the research studies conducted in the field of teacher education, with special reference

More information

Validation of Scales

Validation of Scales Validation of Scales ἀγεωμέτρητος μηδεὶς εἰσίτω (Let none enter without a knowledge of mathematics) D R D I N E S H R A M O O Introduction Validity and validation are crucial to understanding psychological

More information

Child Outcomes Research Consortium. Recommendations for using outcome measures

Child Outcomes Research Consortium. Recommendations for using outcome measures Child Outcomes Research Consortium Recommendations for using outcome measures Introduction The use of outcome measures is one of the most powerful tools available to children s mental health services.

More information

PSYCHOMETRIC PROPERTIES OF CLINICAL PERFORMANCE RATINGS

PSYCHOMETRIC PROPERTIES OF CLINICAL PERFORMANCE RATINGS PSYCHOMETRIC PROPERTIES OF CLINICAL PERFORMANCE RATINGS A total of 7931 ratings of 482 third- and fourth-year medical students were gathered over twelve four-week periods. Ratings were made by multiple

More information

Research Approaches Quantitative Approach. Research Methods vs Research Design

Research Approaches Quantitative Approach. Research Methods vs Research Design Research Approaches Quantitative Approach DCE3002 Research Methodology Research Methods vs Research Design Both research methods as well as research design are crucial for successful completion of any

More information

Project 3: Design a Questionnaire

Project 3: Design a Questionnaire Project 3: Design a Questionnaire Your Task Design a questionnaire to measure a psychological construct of your choice. Many questionnaires exist already that psychologists have tried to use to measure

More information

MN 400: Research Methods CHAPTER 8. Survey Methods: Communication with Participants

MN 400: Research Methods CHAPTER 8. Survey Methods: Communication with Participants MN 400: Research Methods CHAPTER 8 Survey Methods: Communication with Participants 1 Interviewing for Statistical Production What is an Interview? Interviewing can mean a lot What do people think about

More information

Designing Experiments... Or how many times and ways can I screw that up?!?

Designing Experiments... Or how many times and ways can I screw that up?!? www.geo.uzh.ch/microsite/icacogvis/ Designing Experiments... Or how many times and ways can I screw that up?!? Amy L. Griffin AutoCarto 2012, Columbus, OH Outline When do I need to run an experiment and

More information

Chapter 6. Methods of Measuring Behavior Pearson Prentice Hall, Salkind. 1

Chapter 6. Methods of Measuring Behavior Pearson Prentice Hall, Salkind. 1 Chapter 6 Methods of Measuring Behavior 2009 Pearson Prentice Hall, Salkind. 1 CHAPTER OVERVIEW Tests and Their Development Types of Tests Observational Techniques Questionnaires 2009 Pearson Prentice

More information

Free Choice or Forced Choice? Your Choice

Free Choice or Forced Choice? Your Choice Free Choice or Forced Choice? Your Choice Peter Saville, Rab MacIver, Rainer Kurz Liverpool, 15 th July 2008 Normative and Ipsative Normative Questionnaires: The respondent rates their agreement or disagreement

More information

Identifying or Verifying the Number of Factors to Extract using Very Simple Structure.

Identifying or Verifying the Number of Factors to Extract using Very Simple Structure. Identifying or Verifying the Number of Factors to Extract using Very Simple Structure. As published in Benchmarks RSS Matters, December 2014 http://web3.unt.edu/benchmarks/issues/2014/12/rss-matters Jon

More information

Personality and Individual Differences

Personality and Individual Differences Personality and Individual Differences 45 (2008) 655 660 Contents lists available at ScienceDirect Personality and Individual Differences journal homepage: www.elsevier.com/locate/paid Beliefs around luck:

More information

HARRISON ASSESSMENTS DEBRIEF GUIDE 1. OVERVIEW OF HARRISON ASSESSMENT

HARRISON ASSESSMENTS DEBRIEF GUIDE 1. OVERVIEW OF HARRISON ASSESSMENT HARRISON ASSESSMENTS HARRISON ASSESSMENTS DEBRIEF GUIDE 1. OVERVIEW OF HARRISON ASSESSMENT Have you put aside an hour and do you have a hard copy of your report? Get a quick take on their initial reactions

More information

Validity and Reliability. PDF Created with deskpdf PDF Writer - Trial ::

Validity and Reliability. PDF Created with deskpdf PDF Writer - Trial :: Validity and Reliability PDF Created with deskpdf PDF Writer - Trial :: http://www.docudesk.com Validity Is the translation from concept to operationalization accurately representing the underlying concept.

More information

Adult Autism Team NHSGG&C. Dr Gwen Jones Edwards Anne Marie Gallagher

Adult Autism Team NHSGG&C. Dr Gwen Jones Edwards Anne Marie Gallagher Adult Autism Team NHSGG&C Dr Gwen Jones Edwards Anne Marie Gallagher A "perfect storm is an expression that describes an event where a rare combination of circumstances will aggravate a situation drastically.

More information

Chapter 9: Comparing two means

Chapter 9: Comparing two means Chapter 9: Comparing two means Smart Alex s Solutions Task 1 Is arachnophobia (fear of spiders) specific to real spiders or will pictures of spiders evoke similar levels of anxiety? Twelve arachnophobes

More information

International Conference on Humanities and Social Science (HSS 2016)

International Conference on Humanities and Social Science (HSS 2016) International Conference on Humanities and Social Science (HSS 2016) The Chinese Version of WOrk-reLated Flow Inventory (WOLF): An Examination of Reliability and Validity Yi-yu CHEN1, a, Xiao-tong YU2,

More information

Quality of Life Assessment of Growth Hormone Deficiency in Adults (QoL-AGHDA)

Quality of Life Assessment of Growth Hormone Deficiency in Adults (QoL-AGHDA) Quality of Life Assessment of Growth Hormone Deficiency in Adults (QoL-AGHDA) Guidelines for Users May 2016 B1 Chorlton Mill, 3 Cambridge Street, Manchester, M1 5BY, UK Tel: +44 (0)161 226 4446 Fax: +44

More information

Teachers Sense of Efficacy Scale: The Study of Validity and Reliability

Teachers Sense of Efficacy Scale: The Study of Validity and Reliability EUROPEAN ACADEMIC RESEARCH Vol. II, Issue 12/ March 2015 ISSN 2286-4822 www.euacademic.org Impact Factor: 3.1 (UIF) DRJI Value: 5.9 (B+) Teachers Sense of Efficacy Scale: The Study of Validity and Dr.

More information

Psychometric properties of the Chinese quality of life instrument (HK version) in Chinese and Western medicine primary care settings

Psychometric properties of the Chinese quality of life instrument (HK version) in Chinese and Western medicine primary care settings Qual Life Res (2012) 21:873 886 DOI 10.1007/s11136-011-9987-3 Psychometric properties of the Chinese quality of life instrument (HK version) in Chinese and Western medicine primary care settings Wendy

More information

Factor Analysis of Gulf War Illness: What Does It Add to Our Understanding of Possible Health Effects of Deployment?

Factor Analysis of Gulf War Illness: What Does It Add to Our Understanding of Possible Health Effects of Deployment? October 3, 2006 Factor Analysis Examples: Example 1: Factor Analysis of Gulf War Illness: What Does It Add to Our Understanding of Possible Health Effects of Deployment? 1 2 2 Susan E. Shapiro, Michael

More information

Attention and Concentration Problems Following Traumatic Brain Injury. Patient Information Booklet. Talis Consulting Limited

Attention and Concentration Problems Following Traumatic Brain Injury. Patient Information Booklet. Talis Consulting Limited Attention and Concentration Problems Following Traumatic Brain Injury Patient Information Booklet Talis Consulting Limited What are Attention and Concentration? Attention and concentration are two skills

More information

Take new look emotions we see as negative may be our best friends (opposite to the script!)

Take new look emotions we see as negative may be our best friends (opposite to the script!) Tony Robbins - emotions Take new look emotions we see as negative may be our best friends (opposite to the script!) How do people deal? 1. They avoid them try not to feel them. a. Rejection 2. I ll endure

More information

CHAPTER 2. RESEARCH METHODS AND PERSONALITY ASSESSMENT (64 items)

CHAPTER 2. RESEARCH METHODS AND PERSONALITY ASSESSMENT (64 items) CHAPTER 2. RESEARCH METHODS AND PERSONALITY ASSESSMENT (64 items) 1. Darwin s point of view about empirical research can be accurately summarized as... a. Any observation is better than no observation

More information

Reliability, validity, and all that jazz

Reliability, validity, and all that jazz Reliability, validity, and all that jazz Dylan Wiliam King s College London Published in Education 3-13, 29 (3) pp. 17-21 (2001) Introduction No measuring instrument is perfect. If we use a thermometer

More information

VARIABLES AND MEASUREMENT

VARIABLES AND MEASUREMENT ARTHUR SYC 204 (EXERIMENTAL SYCHOLOGY) 16A LECTURE NOTES [01/29/16] VARIABLES AND MEASUREMENT AGE 1 Topic #3 VARIABLES AND MEASUREMENT VARIABLES Some definitions of variables include the following: 1.

More information

Attitude = Belief + Evaluation. TRA/TPB and HBM. Theory of Reasoned Action and Planned Behavior. TRA: Constructs TRA/TPB

Attitude = Belief + Evaluation. TRA/TPB and HBM. Theory of Reasoned Action and Planned Behavior. TRA: Constructs TRA/TPB and HBM Theory of Reasoned Action and Planned Behavior Both focus on rational, cognitive decision-making processes adds the social context to the basic ideas of the HBM 2 TRA: Constructs Behavioral Intention

More information

So far. INFOWO Lecture M5 Homogeneity and Reliability. Homogeneity. Homogeneity

So far. INFOWO Lecture M5 Homogeneity and Reliability. Homogeneity. Homogeneity So far INFOWO Lecture M5 Homogeneity and Reliability Peter de Waal Department of Information and Computing Sciences Faculty of Science, Universiteit Utrecht Descriptive statistics Scores and probability

More information

Chapter 17: Exploratory factor analysis

Chapter 17: Exploratory factor analysis Chapter 17: Exploratory factor analysis Labcoat Leni s Real Research Worldwide addiction? Problem Nichols, L. A., & Nicki, R. (2004). Psychology of Addictive Behaviors, 18(4), 381-384 In 2007 it was estimated

More information

Chapter -6 Reliability and Validity of the test Test - Retest Method Rational Equivalence Method Split-Half Method

Chapter -6 Reliability and Validity of the test Test - Retest Method Rational Equivalence Method Split-Half Method Chapter -6 Reliability and Validity of the test 6.1 Introduction 6.2 Reliability of the test 6.2.1 Test - Retest Method 6.2.2 Rational Equivalence Method 6.2.3 Split-Half Method 6.3 Validity of the test

More information

Chapter 9. Youth Counseling Impact Scale (YCIS)

Chapter 9. Youth Counseling Impact Scale (YCIS) Chapter 9 Youth Counseling Impact Scale (YCIS) Background Purpose The Youth Counseling Impact Scale (YCIS) is a measure of perceived effectiveness of a specific counseling session. In general, measures

More information

Motivational Interviewing

Motivational Interviewing Motivational Interviewing By: Tonia Stott, PhD What is Motivational Interviewing? A client-centered, directive method for enhancing intrinsic motivation to change by exploring and resolving ambivalence

More information

The measurement of media literacy in eating disorder risk factor research: psychometric properties of six measures

The measurement of media literacy in eating disorder risk factor research: psychometric properties of six measures McLean et al. Journal of Eating Disorders (2016) 4:30 DOI 10.1186/s40337-016-0116-0 RESEARCH ARTICLE Open Access The measurement of media literacy in eating disorder risk factor research: psychometric

More information

Experimental Research in HCI. Alma Leora Culén University of Oslo, Department of Informatics, Design

Experimental Research in HCI. Alma Leora Culén University of Oslo, Department of Informatics, Design Experimental Research in HCI Alma Leora Culén University of Oslo, Department of Informatics, Design almira@ifi.uio.no INF2260/4060 1 Oslo, 15/09/16 Review Method Methodology Research methods are simply

More information

PSYC2600 Lecture One Attitudes

PSYC2600 Lecture One Attitudes PSYC2600 Lecture One Attitudes 16/07/14 7:01 AM Motivation (Lei, 2010) Intrinsic motivation have an internal motivation, you are interested. Extrinsic motivation have no specific interest, but are motivated

More information

A report about. Anxiety. Easy Read summary

A report about. Anxiety. Easy Read summary A report about Anxiety Easy Read summary Mental Health Awareness Week 2014 This is an Easy Read summary of the Living with Anxiety report (2014). This report was written by the Mental Health Foundation.

More information

1. Before starting the second session, quickly examine total on short form BDI; note

1. Before starting the second session, quickly examine total on short form BDI; note SESSION #2: 10 1. Before starting the second session, quickly examine total on short form BDI; note increase or decrease. Recall that rating a core complaint was discussed earlier. For the purpose of continuity,

More information

Key words: State-Trait Anger, Anger Expression, Anger Control, FSTAXI-2, reliability, validity.

Key words: State-Trait Anger, Anger Expression, Anger Control, FSTAXI-2, reliability, validity. Psychometric Properties of Farsi version of the Spielberger s State-Trait Anger Expression Inventory-2 (FSTAXI-2) Mohammad Khodayarifard 1, Charles D. Spielberger 2 Masoud Gholamali Lavasani 1, Saeed Akbari

More information

ANSWERS: Research Methods

ANSWERS: Research Methods ANSWERS: Research Methods Advice: Most of these answers will fit in the boxes if writing is small, and students can use continuation sheets wherever necessary. Please note that they are not definitive

More information

ISC- GRADE XI HUMANITIES ( ) PSYCHOLOGY. Chapter 2- Methods of Psychology

ISC- GRADE XI HUMANITIES ( ) PSYCHOLOGY. Chapter 2- Methods of Psychology ISC- GRADE XI HUMANITIES (2018-19) PSYCHOLOGY Chapter 2- Methods of Psychology OUTLINE OF THE CHAPTER (i) Scientific Methods in Psychology -observation, case study, surveys, psychological tests, experimentation

More information

Factor Analysis. MERMAID Series 12/11. Galen E. Switzer, PhD Rachel Hess, MD, MS

Factor Analysis. MERMAID Series 12/11. Galen E. Switzer, PhD Rachel Hess, MD, MS Factor Analysis MERMAID Series 2/ Galen E Switzer, PhD Rachel Hess, MD, MS Ways to Examine Groups of Things Groups of People Groups of Indicators Cluster Analysis Exploratory Factor Analysis Latent Class

More information

She has an extensive psychiatric history, with numerous admissions, and minor selfharm.

She has an extensive psychiatric history, with numerous admissions, and minor selfharm. Instructions for Candidates Scenario Rhona Anderson is a 28-year-old woman who has been referred for assessment by her GP. She had presented to her GP expressing suicidal ideation. She has an extensive

More information

Quantitative Methods in Computing Education Research (A brief overview tips and techniques)

Quantitative Methods in Computing Education Research (A brief overview tips and techniques) Quantitative Methods in Computing Education Research (A brief overview tips and techniques) Dr Judy Sheard Senior Lecturer Co-Director, Computing Education Research Group Monash University judy.sheard@monash.edu

More information

DATA is derived either through. Self-Report Observation Measurement

DATA is derived either through. Self-Report Observation Measurement Data Management DATA is derived either through Self-Report Observation Measurement QUESTION ANSWER DATA DATA may be from Structured or Unstructured questions? Quantitative or Qualitative? Numerical or

More information

The Relationship between YouTube Interaction, Depression, and Social Anxiety. By Meredith Johnson

The Relationship between YouTube Interaction, Depression, and Social Anxiety. By Meredith Johnson The Relationship between YouTube Interaction, Depression, and Social Anxiety By Meredith Johnson Introduction The media I would like to research is YouTube with the effects of social anxiety and depression.

More information

Validity. Ch. 5: Validity. Griggs v. Duke Power - 2. Griggs v. Duke Power (1971)

Validity. Ch. 5: Validity. Griggs v. Duke Power - 2. Griggs v. Duke Power (1971) Ch. 5: Validity Validity History Griggs v. Duke Power Ricci vs. DeStefano Defining Validity Aspects of Validity Face Validity Content Validity Criterion Validity Construct Validity Reliability vs. Validity

More information

Coeliac Disease: Eating Attitudes and Behaviours. Rosie Satherley, Professor Suzanne Higgs, Dr. Ruth Howard University of Birmingham

Coeliac Disease: Eating Attitudes and Behaviours. Rosie Satherley, Professor Suzanne Higgs, Dr. Ruth Howard University of Birmingham Coeliac Disease: Eating Attitudes and Behaviours Rosie Satherley, Professor Suzanne Higgs, Dr. Ruth Howard University of Birmingham Who am I? University of Birmingham, UK Adults with coeliac disease Kings

More information

Evaluating Quality in Creative Systems. Graeme Ritchie University of Aberdeen

Evaluating Quality in Creative Systems. Graeme Ritchie University of Aberdeen Evaluating Quality in Creative Systems Graeme Ritchie University of Aberdeen Graeme Ritchie {2007} Some Empirical Criteria for Attributing Creativity to a Computer Program. Minds and Machines 17 {1}, pp.67-99.

More information

Meeting-5 MEASUREMENT 8-1

Meeting-5 MEASUREMENT 8-1 Meeting-5 MEASUREMENT 8-1 Measurement Measurement Process: 1. Selecting observable empirical events 2. Using numbers or symbols to represent aspects of the events being measured 3. Applying a mapping rule

More information

Measurement. 500 Research Methods Mike Kroelinger

Measurement. 500 Research Methods Mike Kroelinger Measurement 500 Research Methods Mike Kroelinger Levels of Measurement Nominal Lowest level -- used to classify variables into two or more categories. Cases placed in the same category must be equivalent.

More information

What Constitutes a Good Contribution to the Literature (Body of Knowledge)?

What Constitutes a Good Contribution to the Literature (Body of Knowledge)? What Constitutes a Good Contribution to the Literature (Body of Knowledge)? Read things that make good contributions to the body of knowledge. The purpose of scientific research is to add to the body of

More information

Talking to someone who might be suicidal

Talking to someone who might be suicidal Talking to someone who might be suicidal To some it s a tea bag. To others it s a lifeline... Support the Zero Suicide Alliance campaign. Help us tackle the stigma that stops so many from asking for help.

More information

Towards an instrument for measuring sensemaking and an assessment of its theoretical features

Towards an instrument for measuring sensemaking and an assessment of its theoretical features http://dx.doi.org/10.14236/ewic/hci2017.86 Towards an instrument for measuring sensemaking and an assessment of its theoretical features Kholod Alsufiani Simon Attfield Leishi Zhang Middlesex University

More information

Value of emotional intelligence in veterinary practice teams

Value of emotional intelligence in veterinary practice teams Vet Times The website for the veterinary profession https://www.vettimes.co.uk Value of emotional intelligence in veterinary practice teams Author : MAGGIE SHILCOCK Categories : Vets Date : February 17,

More information

Measuring Anxiety Towards Wiki Editing: Investigating the Dimensionality of the Wiki Anxiety Inventory-Editing

Measuring Anxiety Towards Wiki Editing: Investigating the Dimensionality of the Wiki Anxiety Inventory-Editing The Author 2013. Published by Oxford University Press on behalf of The British Computer Society. All rights reserved. For Permissions, please email: journals.permissions@oup.com Advance Access publication

More information

PSI R ESEARCH& METRICS T OOLKIT. Writing Scale Items and Response Options B UILDING R ESEARCH C APACITY. Scales Series: Chapter 3

PSI R ESEARCH& METRICS T OOLKIT. Writing Scale Items and Response Options B UILDING R ESEARCH C APACITY. Scales Series: Chapter 3 PSI R ESEARCH& METRICS T OOLKIT B UILDING R ESEARCH C APACITY Scales Series: Chapter 3 Writing Scale Items and Response Options PSI s Core Values Bottom Line Health Impact * Private Sector Speed and Efficiency

More information

Empowered by Psychometrics The Fundamentals of Psychometrics. Jim Wollack University of Wisconsin Madison

Empowered by Psychometrics The Fundamentals of Psychometrics. Jim Wollack University of Wisconsin Madison Empowered by Psychometrics The Fundamentals of Psychometrics Jim Wollack University of Wisconsin Madison Psycho-what? Psychometrics is the field of study concerned with the measurement of mental and psychological

More information

Communication. Jess Walsh

Communication. Jess Walsh Communication Jess Walsh Introduction. Douglas Bank is a home for young adults with severe learning disabilities. Good communication is important for the service users because it s easy to understand the

More information

Reliability AND Validity. Fact checking your instrument

Reliability AND Validity. Fact checking your instrument Reliability AND Validity Fact checking your instrument General Principles Clearly Identify the Construct of Interest Use Multiple Items Use One or More Reverse Scored Items Use a Consistent Response Format

More information

Statistics is the science of collecting, organizing, presenting, analyzing, and interpreting data to assist in making effective decisions

Statistics is the science of collecting, organizing, presenting, analyzing, and interpreting data to assist in making effective decisions Readings: OpenStax Textbook - Chapters 1 5 (online) Appendix D & E (online) Plous - Chapters 1, 5, 6, 13 (online) Introductory comments Describe how familiarity with statistical methods can - be associated

More information

Introduction to Survey Research. Clement Stone. Professor, Research Methodology.

Introduction to Survey Research. Clement Stone. Professor, Research Methodology. Clement Stone Professor, Research Methodology Email: cas@pitt.edu 1 Presentation Outline What is survey research and when is it used? Stages of survey research 1. Specifying research questions, target

More information

CHAPTER III RESEARCH METHOD. method the major components include: Research Design, Research Site and

CHAPTER III RESEARCH METHOD. method the major components include: Research Design, Research Site and CHAPTER III RESEARCH METHOD This chapter presents the research method and design. In this research method the major components include: Research Design, Research Site and Access, Population and Sample,

More information

CHAPTER 3 METHODOLOGY, DATA COLLECTION AND DATA ANALYSIS

CHAPTER 3 METHODOLOGY, DATA COLLECTION AND DATA ANALYSIS CHAPTER 3 METHODOLOGY, DATA COLLECTION AND DATA ANALYSIS TABLE OF CONTENTS Titles Page Nos. CHAPTER 3: METHODOLOGY, DATA COLLECTION 88 97 AND DATA ANALYSIS 3.1 Objectives of the Study 89 3.2 Research Design

More information

These questionnaires are used by psychology services to help us understand how people feel. One questionnaire measures how sad people feel.

These questionnaires are used by psychology services to help us understand how people feel. One questionnaire measures how sad people feel. ADAPTED PHQ-9 & GAD-7 QUESTIONNAIRES How to fill in these questionnaires: These questionnaires are used by psychology services to help us understand how people feel. One questionnaire measures how sad

More information

Review and Wrap-up! ESP 178 Applied Research Methods Calvin Thigpen 3/14/17 Adapted from presentation by Prof. Susan Handy

Review and Wrap-up! ESP 178 Applied Research Methods Calvin Thigpen 3/14/17 Adapted from presentation by Prof. Susan Handy Review and Wrap-up! ESP 178 Applied Research Methods Calvin Thigpen 3/14/17 Adapted from presentation by Prof. Susan Handy Final Proposals Read instructions carefully! Check Canvas for our comments on

More information

Validity refers to the accuracy of a measure. A measurement is valid when it measures what it is suppose to measure and performs the functions that

Validity refers to the accuracy of a measure. A measurement is valid when it measures what it is suppose to measure and performs the functions that Validity refers to the accuracy of a measure. A measurement is valid when it measures what it is suppose to measure and performs the functions that it purports to perform. Does an indicator accurately

More information

Personality Traits Effects on Job Satisfaction: The Role of Goal Commitment

Personality Traits Effects on Job Satisfaction: The Role of Goal Commitment Marshall University Marshall Digital Scholar Management Faculty Research Management, Marketing and MIS Fall 11-14-2009 Personality Traits Effects on Job Satisfaction: The Role of Goal Commitment Wai Kwan

More information