Unproctored Internet-Based Tests of Cognitive Ability and Personality: Magnitude of Cheating and Response Distortion
|
|
- Cleopatra McGee
- 5 years ago
- Views:
Transcription
1 Industrial and Organizational Psychology, 2 (2009), Copyright ª 2009 Society for Industrial and Organizational Psychology /09 Unproctored Internet-Based Tests of Cognitive Ability and Personality: Magnitude of Cheating and Response Distortion WINFRED ARTHUR, JR., RYAN M. GLAZE, AND ANTON J. VILLADO Texas A&M University JASON E. TAYLOR People Answers, Inc. Malfeasant behaviors on tests may take one of two forms cheating or response distortion. Cheating is associated with ability and knowledge tests, and response distortion is associated with noncognitive/personality measures. Pursuant to the cheating and response distortion-related questions posed by Tippins (2009), the objective of the present article is to present a summary of the pertinent results of a study (i.e., Arthur, Glaze, Villado, & Taylor, 2008) that was designed to (a) investigate the efficacy of using a speeded test format to address concerns about cheating on an unproctored Internetbased cognitive ability test and (b) document the extent to which the magnitude of response distortion on two unproctored Internet testing (UIT) personality measures was similar to or different from that reported Correspondence concerning this article should be addressed to Winfred Arthur, Jr. w-arthur@ tamu.edu Address: Department of Psychology, Texas A&M University, 4235 TAMU, College Station, TX Winfred Arthur, Department of Psychology, Texas A&M University; Ryan M. Glaze and Anton J. Villado, Department of Psychology, Texas A&M University; Jason E. Taylor, People Answers, Inc. for proctored personality measures in the extant literature. Cheating on Cognitive Ability Tests It is not unreasonable to assume that cheating can and does occur in employment testing. Although we were unable to locate any published empirical studies on the prevalence of cheating in employment testing, the prevalence of cheating in educational settings seems to be quite widespread. For example, reviews by Cizek (1999) and Whitley (1998) indicate that approximately half of all college students reported cheating on an exam at least once during their college education. Furthermore, it seems reasonable that the prevalence of cheating is likely to be exacerbated with the use of UIT because proctoring is the primary means by which cheating is curtailed. Several techniques to prevent and detect cheating are noted by Tippins. As yet another potential technique, Arthur et al. (2009) investigated the efficacy of the use of speeded tests that, by virtue of their time constraints, may serve to curtail the anticipated cheating behaviors. Of course, this is predicated on the assumption that a speeded administration is consonant 39
2 40 W. Arthur et al. with the job-relatedness of the test. Assuming there is no preknowledge of test content, possible modes of cheating (e.g., using additional aids or helpers) are not independent of time. That is, if individuals do not have a preknowledge of the test content, and are not using surrogate test takers, then time constraints should make it more difficult to and subsequently deter them from engaging in malfeasant behaviors under speeded conditions. Consequently, an objective of Arthur et al. was to investigate whether observed retest score changes on a speeded UIT cognitive ability test (120 items with a 20-minute administration time) could be best accounted for by a cheating or psychometric retest explanation. In addition, Arthur et al. also investigated whether the speeded UIT cognitive ability test differed from proctored cognitive ability tests (in the extant literature) in terms of retest score changes. Response Distortion on Personality Tests As previously noted, the primary malfeasant behavior of interest with personality and other noncognitive measures is response distortion. Although there is no clear consensus about their effectiveness and efficacy, several techniques have been proposed for preventing or minimizing response distortion on personality measures. These include the use of forced-choice responses, empirical keying, warnings of verification, and response elaboration (e.g., see, Hough, 1998 for a review). However, absent from this list of techniques is the use of test proctors because the presence of test proctors has little or no bearing on controlling response distortion. Consequently, in contrast to cognitive ability (and knowledge) tests, concerns regarding response distortion on personality and other noncognitive tests are the same in both proctored and unproctored testing conditions. In summary, it is not unreasonable to expect that the magnitude of response distortion should be similar for both UIT and proctored personality tests. Hence, a second objective of Arthur et al. (2009) was to investigate whether unproctored and proctored personality tests (based on the extant literature) differed in terms of the magnitude of response distortion specifically, do UIT personality tests display higher levels of response distortion than proctored tests? Summary of Arthur et al. s (2009) Results Arthur et al. accomplished their study objectives by implementing two within-subjects design studies (Study 1, N ¼ 296; Study 2, N ¼ 318) in which test takers first completed UIT cognitive ability and personality measures as job applicants (high stakes) or incumbents (low stakes), then as research participants (low stakes). Thus, applicants were considered to have experienced highstakes testing and low-stakes retesting (i.e., HL-stakes) and incumbents were posited to have experienced low stakes during both testing and retesting (i.e., LL-stakes). Because the testing firm uses the tests for a wide range of positions for their clients, Arthur et al. s participants represented a variety of jobs (17 Standard Occupational Classification [SOC] major groups were represented) in a number of organizations (14 North American Industry Classification System [NAICS] industry types were represented). Across both studies, the three most common SOC groups were management occupations, business and finance occupations, and sales and related occupations, and for the NAICS industry types, the three most common were retail trade; finance and insurance; and professional, scientific, and technical services. In Study 1, participants completed both the cognitive ability and the personality measures. However, in Study 2, participants completed only a personality measure (which was different from that used in Study 1), and in addition, there were no Time 1 incumbents in Study 2. For the proctored ability and personality test comparisons, Arthur et al. compared their results to the meta-analytic results reported by Hausknecht, Halpert, Di Paolo, and Moriarty- Gerrard (2007) and Birkeland, Manson, Kisamore, Brannick, and Smith (2006),
3 Unproctored Internet testing 41 respectively. The pertinent findings of these meta-analyses are briefly reviewed below. The results of Hausknecht et al. s (2007) meta-analysis of practice effects on proctored cognitive ability tests indicated retest improvements in scores both in the presence (d ¼ 0.64, k ¼ 23, N ¼ 2,323) and in the absence (d ¼ 0.21, k ¼ 75, N ¼ 81,374) of interventions such as test coaching and training. In addition, the mean improvement in test scores under operational (d ¼ 0.27, k ¼ 19, N ¼ 61,795) and research-based testing conditions (d ¼ 0.22, k ¼ 88, N ¼ 72,641) was quite similar. It should be noted that unlike Arthur et al. (2009), the testing conditions for the test and retest in Hausknecht et al. were identical. Thus, for the operational setting, both conditions were high stakes, and likewise for the research conditions, both were low stakes. For the issues at hand, the operational results are most relevant. Birkeland et al. (2006) investigated job applicant response distortion (faking) on personality measures by comparing applicants and incumbents scores on the five-factor model (FFM) of personality dimensions. Birkeland et al. also drew the distinction between indirect and direct measures of the FFM dimensions where direct measures (e.g., the NEO-FFI) were defined as those that were specifically designed to measure the FFM personality factors. In contrast, indirect measures (e.g., the 16PF) were not specifically designed to measure the FFM personality factors but could and were sorted into the FFM personality dimensions. As a summary statement, Birkeland et al. s results generally indicated higher mean scores for applicants. However, their results also indicated that the magnitude of the mean differences between applicants and incumbents was a function of the personality dimension and the test type (i.e., direct vs. indirect measure). Although Arthur et al. (2009) used indirect FFM measures, their results are compared to both Birkeland et al. s indirect and direct measure results. The results of Arthur et al. s (2009) studies are summarized in Tables 1 and 2 and Figures 1 and 2. For Figures 1 and 2, Arthur et al. used the standard error of measurement (SEM) of the difference scores (i.e., 1 SEM d ) to identify individuals who may have engaged in malfeasance. A number of summary statements can be made on the basis of their results. First, the use of a speeded UIT cognitive ability time resulted in high-stakes/ low-stakes retest effects that were more consonant with a psychometric practice effect than a malfeasance explanation. Specifically, consistent with psychometric theory, the Time 2 scores were moderately higher than the Time 1 scores (d ¼ 0.39). These findings are consistent with those reported by Hausknecht et al. (2007). Furthermore, using the 1 SEM d to operationalize malfeasance (i.e., differences between Time 2 and Time 1 scores that fell below the 1 SEM d band [Time 2 score was lower than the Time 1 score] were considered to be evidence of malfeasance) indicated that there was a relatively small percentage of cheaters (7.77%). 1 In interpreting these data, it should be noted that lower Time 2 scores (which we infer to be cheating ) could be because of (a) the Time 1 score being elevated because of cheating and the Time 2 score being the true score or (b) the Time 1 score being the true score and Time 2 score being an unmotivated test performance score (i.e., the participant did not take the test seriously). Given their design and data, Arthur et al. were unable to distinguish or differentiate between these two causes or explanations of the observed score difference. The pattern of results and associated conclusions of Arthur et al. are consistent with those of Nye, Do, Drasgow, and Fine (2008) who used a perceptual speed test. Consequently, the percentage of cheaters in Arthur et al. s sample conceivably ranges from 0 (all lower Time 2 scores are because of explanation 2) to 7.7% (all lower Time 2 scores are because of explanation 1). Hence, 7.7% might be best viewed as the upper limit for cheating in their sample. So in their totality, their 1. Although not summarized here, Arthur et al. (2008) also present data on the distributional placement (i.e., position in the score distribution) of malfeasant responders on both the cognitive ability and the personality measures. These data are available upon request or can be obtained from the article.
4 42 W. Arthur et al. Table 1. Comparison of Arthur et al. s (2009) Study 1 Cognitive Ability (Unproctored) Test Retest Standardized Mean Differences (d) and Retest Reliability Coefficients With Hausknecht et al. s (2007) Meta-Analytic Results Arthur et al. Study 1 (unproctored) Hausknecht et al. (proctored) Total HL LL Op a Rsch b Cognitive ability 0.39* 0.36* 0.57* Time 22Time 1 d Retest reliability c Note. N ¼ 296, HL n ¼ 252, LL n ¼ 44. ds were computed by subtracting the Time 1 scores from the Time 2 scores so a positive d indicates that the Time 2 score is greater than the Time 1 score. a Operational data. b Research-based data. c Hausknecht et al. do not present an operational/research retest reliability breakdown, and thus, the reliability estimate is for both settings. For comparative purposes, Arthur et al. s mean retest interval was days (SD ¼ 54.84, median ¼ ); in contrast, Hausknecht et al. report a mean of days (SD ¼ , median ¼ 20.00). *p,.05, two-tailed. results did not support the presence of widespread score increases on the speeded ability test as a result of high-stakes testing. Consequently, it would seem that the use of speeded tests, assuming they are not at odds with the job requirements, might be one means of alleviating cheating concerns with UIT ability tests. Second, because proctoring is not a technique that is intended to prevent or minimize response distortion on noncognitive measures, it was expected that UIT personality measures would display levels of response distortion similar to those reported for proctored measures in the extant literature (e.g., Birkeland et al., 2006). The results of both of Arthur et al. s (2008) studies supported this supposition. Thus, like proctored measures, FFM dimension scores were generally higher in the high-stakes condition than the lowstakes. In addition, the retest reliability coefficients were similar in magnitude and range to those reported for proctored tests using similar designs (e.g., see Ellingson, Sackett,& Connelly, 2007, Table 5; Hogan, Barrett, & Hogan, 2007, Tables 2 & 3). Furthermore, as Table 2. Comparison of Arthur et al. s (2009) Study 1 and Study 2 Personality Factors (Unproctored) Test Retest Standardized Mean Differences (d) and Retest Reliability Coefficients With Birkeland et al. s (2006) Meta-Analytic Results Five-factor model dimensions Arthur et al. Study 1 (unproctored) Arthur et al. Study 2 (unproctored) Birkeland et al. (proctored) Total HL LL HL Direct Indirect d r xx d r xx d r xx d r xx d d Agreeableness 20.63* * * * Conscientiousness 20.66* * * * Emotional stability 20.63* * * Extraversion 20.29* * * Openness 20.20* * Note. Study 1, N ¼ 296 (HL n ¼ 252, LL n ¼ 44). Study 2, N ¼ 318. ds were computed by subtracting the Time 1 scores from the Time 2 scores so a positive d indicates that the Time 2 score is greater than the Time 1 score. The mean retest interval for Study 1 was days (SD ¼ 54.84, median ¼ ), and for Study 2, the mean was days (SD ¼ 35.84, median ¼ ). *p,.05, two-tailed.
5 Unproctored Internet testing % 90% 80% Percentage of Test Takers 70% 60% 50% 40% 30% 49.32% 42.91% 20% 10% 7.77% 0% Evidence of Practice Effect Evidence of Stability Evidence of Cheating Score Difference Categories Figure 1. Percentage of Arthur et al. (2008) Study 1 test takers as per 1 SEM d operationalization of practice and cheating effects on the cognitive ability test. with proctored tests, the magnitude of the score shifts was generally higher for Agreeableness, Conscientiousness, and Emotional Stability compared to Extraversion and Openness. However, it should be noted that compared to Birkeland et al. s results for indirect measures, the retest effects for Arthur et al. s data tended to be slightly larger a pattern of results that is consistent with the finding (see Edens & Arthur, 2000; Viswesvaran & Ones, 1999) that larger response distortion effects are generally obtained for within-subject designs (Arthur et al. s data) than betweensubject designs (Birkeland et al. s data). Finally, the percentages of individuals identified as having elevated applicant scores were similar to those reported by Griffith, Chmielowski, and Yoshita (2007) for proctored tests, that is 30 50%. In summary, on the basis of Arthur et al. s (2009) data, one could conclude that UIT personality measures display mean score shifts between high- versus low-stakes testing conditions that are similar to those reported for proctored personality measures. In contrast, we suspect that if the cognitive ability test used in Arthur et al. s study had not been speeded, therefore increasing opportunities to engage in malfeasant behaviors, it would have displayed test scores in the high-stakes condition that would have been higher than those reported for proctored ability tests. However, it would seem that the use of a speeded test resulted in retest effects that were similar to those reported for proctored ability tests in the extant literature. Conclusions The efficacy of UIT employment testing is threatened by the possible influence of malfeasance on the part of applicants. However, we review the results of a study that suggests that even under conditions where it is very intuitive to expect widespread cheating and malfeasance (i.e., high-stakes unproctored ability testing), there was no discernable effect on test scores when the testing condition was designed to counter such behavior. The UIT using a speeded test format appears to have reduced the opportunity for and thus the prevalence of malfeasant behavior. However, although they may curtail
6 44 W. Arthur et al. 100% Study 1 90% 80% Percentage of Test Takers 70% 60% 50% 40% 30% 20% 10% 0% 64.86% 62.84% 59.80% 59.80% 51.35% 35.81% 34.12% 33.11% 35.81% 25.68% 12.84% 14.53% 4.39% 3.04% 2.03% Agreeableness Conscientiousness Emotional Stability Extraversion Openness Evidence of Response Distortion (Low) Evidence of Stability Evidence of Response Distortion (High) 100% Study 2 90% 80% Percentage of Test Takers 70% 60% 50% 40% 30% 20% 10% 0% 65.72% 63.52% 58.49% 56.60% 49.06% 34.59% 36.16% 32.39% 22.96% 18.55% 21.70% 14.78% 11.32% 6.92% 7.23% Agreeableness Conscientiousness Emotional Stability Extraversion Openness Evidence of Response Distortion (Low) Evidence of Stability Evidence of Response Distortion (High) Figure 2. Percentage of Arthur et al. (2009) Study 1 and Study 2 test takers as per 1 SEM d operationalization of response distortion on the personality measure. cheating-related behaviors that are temporally resource demanding, they may do little to preempt the use of surrogate test takers. Nevertheless, where appropriate, the use of speeded UIT ability tests may be an additional option or alternative to on-site retesting. Similarly, Arthur et al. s results suggest that using a UIT administration does not uniquely threaten personality measures in terms of elevated levels of response
7 Unproctored Internet testing 45 distortion that are higher than those observed for proctored tests. Specifically, the pattern of high-stakes versus low-stakes retest effects observed for UIT and proctored personality measures was quite similar. References Arthur, W., Jr., Glaze, R. M., Villado, A. J., & Taylor, J. E. (2009). The magnitude and extent of cheating and response distortion effects on unproctored internetbased tests of cognitive ability and personality. Manuscript submitted for publication. Birkeland, S. A., Manson, T. M., Kisamore, J. L., Brannick, M. T., & Smith, M. A. (2006). A meta-analytic investigation of job applicant faking on personality measures. International Journal of Selection and Assessment, 14, Cizek, G. J. (1999). Cheating on tests: How to do it, detect it, and prevent it. Mahwah, NJ: Lawrence Erlbaum Associates. Edens, P. S., & Arthur, W., Jr. (2000). A meta-analysis investigating the susceptibility of self-report inventories to distortion. Paper presented at the 15th annual conference of the Society for Industrial and Organizational Psychology, New Orleans, LA. Ellingson, J. E., Sackett, P. R., & Connelly, B. S. (2007). Personality assessment across selection and development contexts: Insights into response distortion. Journal of Applied Psychology, 92, Griffith, R. L., Chmielowski, T., & Yoshita, Y. (2007). Do applicants fake? An examination of the frequency of applicant faking behavior. Personnel Review, 36, Hausknecht, J. P., Halpert, J. A., Di Paolo, N. T., & Moriarty-Gerrard, M. O. (2007). Retesting in selection: A meta-analysis of coaching and practice effects for tests of cognitive ability. Journal of Applied Psychology, 92, Hogan, J., Barrett, P., & Hogan, R. (2007). Personality measurement, faking, and employment selection. Journal of Applied Psychology, 92, Hough, L. M. (1998). Effects of intentional distortion in personality measurement and evaluation of suggested palliatives. Human Performance, 11, Nye, C. D., Do, B., Drasgow, F., & Fine, S. (2008). Twostep testing in employee selection: Is score inflation a problem? International Journal of Selection and Assessment, 16, Tippins, N. T. (2009). Internet alternatives to traditional proctored testing: Where are we now? Industrial and Organizational Psychology: Perspectives on Science and Practice, 2, Viswesvaran, C., & Ones, D. S. (1999). Meta-analyses of fakability estimates: Implications for personality measurement. Educational and Psychological Measurement, 59, Whitley, B. E. (1998). Factors associated with cheating among college students: A review. Research in Higher Education, 39,
Are there personality traits that predispose applicants to fake noncognitive measures in personnel selection?
Review of Psychology, Original scientific paper 2012, Vol. 19, No. 1, 29-36 UDC 159.923.3 Are there personality traits that predispose applicants to fake noncognitive measures in personnel selection? MAŠA
More informationRunning Head: Academic Dishonesty and Applicant Dishonesty. Richard L. Frei. Community College of Philadelphia
Faking and Academic Dishonesty 1 Running Head: Academic Dishonesty and Applicant Dishonesty Exploring the Relationship Between Academic Dishonesty and Applicant Dishonesty Richard L. Frei Community College
More informationPersonality Assessment Across Selection and Development Contexts: Insights Into Response Distortion
Journal of Applied Psychology Copyright 2007 by the American Psychological Association 2007, Vol. 92, No. 2, 386 395 0021-9010/07/$12.00 DOI: 10.1037/0021-9010.92.2.386 Personality Assessment Across Selection
More informationExtraversion. The Extraversion factor reliability is 0.90 and the trait scale reliabilities range from 0.70 to 0.81.
MSP RESEARCH NOTE B5PQ Reliability and Validity This research note describes the reliability and validity of the B5PQ. Evidence for the reliability and validity of is presented against some of the key
More informationRunning head: Race Differences in Personality. Race Differences in Personality: An Evaluation of Moderators and Publication Bias. Brian W.
Running head: Race Differences in Personality Race Differences in Personality: An Evaluation of Moderators and Publication Bias Brian W. Tate Pennsylvania State University Michael A. McDaniel Virginia
More informationWHITE PAPER. A Comparison Between The Birkman Method and The Hogan Personality Inventory
WHITE PAPER A Comparison Between The Birkman Method and The Hogan Personality Inventory Table of Contents I. Introduction....... 2 II. Development.. 2 III. Measurement.. 4 IV. Reliability.... 6 V. Validity...
More informationThe Impact of Corrections for Faking on the Validity of Noncognitive Measures in Selection Settings
Journal of Applied Psychology Copyright 2006 by the American Psychological Association 2006, Vol. 91, No. 3, 613 621 0021-9010/06/$12.00 DOI: 10.1037/0021-9010.91.3.613 The Impact of Corrections for Faking
More informationDetecting Suspect Examinees: An Application of Differential Person Functioning Analysis. Russell W. Smith Susan L. Davis-Becker
Detecting Suspect Examinees: An Application of Differential Person Functioning Analysis Russell W. Smith Susan L. Davis-Becker Alpine Testing Solutions Paper presented at the annual conference of the National
More informationWill Exploring the Darkness Prove Enlightening? Five Questions About the Maladaptive Big Five. JUSTIN A. DESIMONE University of Nebraska-Lincoln
126 J.A. DeSimone Will Exploring the Darkness Prove Enlightening? Five Questions About the Maladaptive Big Five JUSTIN A. DESIMONE University of Nebraska-Lincoln Correspondence concerning this article
More informationCraft Personality Questionnaire
Craft Personality Questionnaire Evidence of Reliability and Validity 888-298-6227 TalentLens.com Copyright 2008 Pearson Education, Inc., or its affiliates. All rights reserved. Copyright 2008 by Pearson
More informationBig Five FFM Five-Factor Model. 1 Neuroticism 3 (Openness to experience) O + O - 4 (Agreeableness) 5 (Conscientiousness) N E O A C
2004 12 4 578~ 583 Advances in Psychological Science 310028 5. B849:C93 Big Five FFM Five-Factor Model 16PF EPQ 5 1949 Fiske Costa McCrae [1] 1 1.1 5 5 1 Neuroticism N + 2Extraversion 3 (Openness to experience)
More informationCHAPTER 2. RESEARCH METHODS AND PERSONALITY ASSESSMENT (64 items)
CHAPTER 2. RESEARCH METHODS AND PERSONALITY ASSESSMENT (64 items) 1. Darwin s point of view about empirical research can be accurately summarized as... a. Any observation is better than no observation
More informationPersonality measures under focus: The NEO-PI-R and the MBTI
: The NEO-PI-R and the MBTI Author Published 2009 Journal Title Griffith University Undergraduate Psychology Journal Downloaded from http://hdl.handle.net/10072/340329 Link to published version http://pandora.nla.gov.au/tep/145784
More informationACADEMIC DISHONESTY, PERSONALITY TRAITS AND ACADEMIC ADJUSTMENT
Bulletin of the Transilvania University of Braşov - Special Issue Series VII: Social Sciences Law Vol. 10 (59) No. 2-2017 ACADEMIC DISHONESTY, PERSONALITY TRAITS AND ACADEMIC ADJUSTMENT A. -M. CAZAN 1
More informationRetesting in Selection: A Meta-Analysis of Practice Effects for Tests of Cognitive Ability
Cornell University ILR School DigitalCommons@ILR Articles and Chapters ILR Collection 6-1-2006 Retesting in Selection: A Meta-Analysis of Practice Effects for Tests of Cognitive Ability John P. Hausknecht
More informationCLINICAL VS. BEHAVIOR ASSESSMENT
CLINICAL VS. BEHAVIOR ASSESSMENT Informal Tes3ng Personality Tes3ng Assessment Procedures Ability Tes3ng The Clinical Interview 3 Defining Clinical Assessment The process of assessing the client through
More informationEthical Perception from Students Perspective: Understanding Instructors Effect on Students Ethical Sensitivity in Personal Selling
Ethical Perception from Students Perspective: Understanding Instructors Effect on Students Ethical Sensitivity in Personal Selling Emrah Cengiz Istanbul University Selim Yazici Istanbul University Murat
More informationFAKING IN PERSONALITY ASSESSMENTS: AN INVESTIGATION OF A METHOD FACTOR MEASURE OF FAKING. Raven Lynnette Worthy
FAKING IN PERSONALITY ASSESSMENTS: AN INVESTIGATION OF A METHOD FACTOR MEASURE OF FAKING By Raven Lynnette Worthy Approved: Michael Biderman Professor of Psychology (Director of Thesis) Brian O Leary Associate
More informationStop Interrupting Me! Examining the Relationship Between Interruptions, Test Performance and Reactions
Volume 3 Issue 1 Article 2 5-31-2017 Stop Interrupting Me! Examining the Relationship Between Interruptions, Test Performance and Reactions Amie D. Lawrence Select International, Inc., alawrence@selectintl.com
More informationEffects of Response Instructions on Faking a Situational Judgment Test
INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT VOLUME 13 NUMBER 4 DECEMBER 2005 Effects of Response Instructions on Faking a Situational Judgment Test Nhung T. Nguyen* Towson University MichaelD.Biderman
More informationEmotional Intelligence and Leadership
The Mayer Salovey Caruso Notes Emotional Intelligence Test (MSCEIT) 2 The Mayer Salovey Caruso Emotional Intelligence Test (MSCEIT) 2 The MSCEIT 2 measures four related abilities. 3 Perceiving Facilitating
More informationAn important aspect of assessment instruments is their stability across time, often called
OVERVIEW The reliability of TRACOM s SOCIAL STYLE Model and SOCIAL STYLE assessments is the focus of this whitepaper. It specifically looks at retest reliability over time including each of the three components
More informationStrategies for Reducing Adverse Impact. Robert E. Ployhart George Mason University
Strategies for Reducing Adverse Impact Robert E. Ployhart George Mason University Overview Diversity, Validity, and Adverse Impact Strategies for Reducing Adverse Impact Conclusions Tradeoff Between Optimal
More informationWork Personality Index Factorial Similarity Across 4 Countries
Work Personality Index Factorial Similarity Across 4 Countries Donald Macnab Psychometrics Canada Copyright Psychometrics Canada 2011. All rights reserved. The Work Personality Index is a trademark of
More informationRunning head: Situational Judgment Item Validity. Toward an Understanding of Situational Judgment Item Validity. Michael A.
Situational judgment item validity 1 Running head: Situational Judgment Item Validity Toward an Understanding of Situational Judgment Item Validity Michael A. McDaniel Virginia Commonwealth University
More informationGENERALIZABILITY AND RELIABILITY: APPROACHES FOR THROUGH-COURSE ASSESSMENTS
GENERALIZABILITY AND RELIABILITY: APPROACHES FOR THROUGH-COURSE ASSESSMENTS Michael J. Kolen The University of Iowa March 2011 Commissioned by the Center for K 12 Assessment & Performance Management at
More informationDEPARTMENT OF INDUSTRIAL PSYCHOLOGY AND PEOPLE MANAGEMENT NOVEMBER EXAM 2014 MS N MORTON
DEPARTMENT OF INDUSTRIAL PSYCHOLOGY AND PEOPLE MANAGEMENT PROGRAMME IN INDUSTRIAL PSYCHOLOGY NOVEMBER EXAM 2014 COURSE: INDUSTRIAL PSYCHOLOGY 2B MODULE CODE: IPS22B2 DATE: 20 NOVEMBER 2014 DURATION: 3
More informationThe Effects of Faking on the Construct Validity of Personality Questionnaires: A Direct Faking Measure Approach
Psychological Topics 21 (2012), 3, 1-18 Original scientific paper UDC 159.923.3.072 The Effects of Faking on the Construct Validity of Personality Questionnaires: A Direct Faking Measure Approach Maša
More informationRelationship between Transformational Leadership and Work-Related Flow in an Industrial Organization
IPA International Journal of Psychology Vol. 6, No. 2, Summer & Fall 2012 PP. 82-98 Iranian Psychological Association Relationship between Transformational Leadership and Work-Related Flow in an Industrial
More informationMultiple Act criterion:
Common Features of Trait Theories Generality and Stability of Traits: Trait theorists all use consistencies in an individual s behavior and explain why persons respond in different ways to the same stimulus
More informationThe Big Five and HEXACO questionnaires are full of affect and so, probably, are your questionnaires
The Big Five and HEXACO questionnaires are full of affect and so, probably, are your questionnaires Michael D. Biderman, Ph.D. The University of Tennessee at Chattanooga pptx at www.utc.edu/faculty/michael-biderman
More informationTELL ME SOME MORE: EXPLORING HOW VERBAL ABILITY AND ITEM VERIFIABILITY INFLUENCE RESPONSES TO BIODATA QUESTIONS IN A HIGH-STAKES SELECTION CONTEXT
PERSONNEL PSYCHOLOGY 2012, 65, 359 383 TELL ME SOME MORE: EXPLORING HOW VERBAL ABILITY AND ITEM VERIFIABILITY INFLUENCE RESPONSES TO BIODATA QUESTIONS IN A HIGH-STAKES SELECTION CONTEXT JULIA LEVASHINA
More informationItem Analysis Explanation
Item Analysis Explanation The item difficulty is the percentage of candidates who answered the question correctly. The recommended range for item difficulty set forth by CASTLE Worldwide, Inc., is between
More informationAdaptive Testing With the Multi-Unidimensional Pairwise Preference Model Stephen Stark University of South Florida
Adaptive Testing With the Multi-Unidimensional Pairwise Preference Model Stephen Stark University of South Florida and Oleksandr S. Chernyshenko University of Canterbury Presented at the New CAT Models
More informationStructural equation models of faking ability in repeated measures designs. Michael D. Biderman
Structural equation models of faking ability in repeated measures designs Michael D. Biderman University of Tennessee at Chattanooga Department of Psychology 615 McCallie Ave., Chattanooga, TN 37403 Tel.:
More informationSaville Consulting Wave Professional Styles Handbook
Saville Consulting Wave Professional Styles Handbook PART 4: TECHNICAL Chapter 19: Reliability This manual has been generated electronically. Saville Consulting do not guarantee that it has not been changed
More informationRETEST EFFECTS IN OPERATIONAL SELECTION SETTINGS: DEVELOPMENT AND TEST OF A FRAMEWORK
PERSONNEL PSYCHOLOGY 2005, 58, 981 1007 RETEST EFFECTS IN OPERATIONAL SELECTION SETTINGS: DEVELOPMENT AND TEST OF A FRAMEWORK FILIP LIEVENS, TINE BUYSE Ghent University, Belgium PAUL R. SACKETT University
More informationBrenton M. Wiernik University of Minnesota. Supplemental results for Study 1
Online Supplemental Material Wiernik, B. M. (2015) Intraindividual personality profiles associated with Realistic interests. Journal of Career Assessment. Brenton M. Wiernik University of Minnesota Supplemental
More informationA review and critique of emotional intelligence measures
Journal of Organizational Behavior J. Organiz. Behav. 26, 433 440 (2005) Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/job.319 A review and critique of emotional intelligence
More informationA closer look at the frame-of-reference effect in personality scale scores and validity
Singapore Management University Institutional Knowledge at Singapore Management University Research Collection Lee Kong Chian School Of Business Lee Kong Chian School of Business 3-2008 A closer look at
More informationRELIABILITY RANKING AND RATING SCALES OF MYER AND BRIGGS TYPE INDICATOR (MBTI) Farida Agus Setiawati
RELIABILITY RANKING AND RATING SCALES OF MYER AND BRIGGS TYPE INDICATOR (MBTI) Farida Agus Setiawati faridaagus@yahoo.co.id. Abstract One of the personality models is Myers Briggs Type Indicator (MBTI).
More informationPredicting integrity with the HEXACO personality model: Use of self- and observer reports
147 Journal of Occupational and Organizational Psychology (2008), 81, 147 167 q 2008 The British Psychological Society The British Psychological Society www.bpsjournals.co.uk Predicting integrity with
More informationThe Relationship of Trait EI with Personality, IQ and Sex in a UK Sample of Employees
The Relationship of Trait EI with Personality, IQ and Sex in a UK Sample of Employees Adriane Arteche*, Tomas Chamorro-Premuzic*, Adrian Furnham** and John Crump*** *Goldsmiths, University of London, London,
More informationEmpowered by Psychometrics The Fundamentals of Psychometrics. Jim Wollack University of Wisconsin Madison
Empowered by Psychometrics The Fundamentals of Psychometrics Jim Wollack University of Wisconsin Madison Psycho-what? Psychometrics is the field of study concerned with the measurement of mental and psychological
More informationChapter 1 Introduction to I/O Psychology
Chapter 1 Introduction to I/O Psychology 1. I/O Psychology is a branch of psychology that in the workplace. a. treats psychological disorders b. applies the principles of psychology c. provides therapy
More informationWORKPLACE FRIENDSHIPS: ORIGINS AND CONSEQUENCES FOR MANAGERIAL EFFECTIVENESS
WORKPLACE FRIENDSHIPS: ORIGINS AND CONSEQUENCES FOR MANAGERIAL EFFECTIVENESS HILLA DOTAN Recanati Graduate School of Business Administration Tel-Aviv University Ramat Aviv, Tel-Aviv 69978, Israel ABSTRACT
More informationAn Empirical Study on Causal Relationships between Perceived Enjoyment and Perceived Ease of Use
An Empirical Study on Causal Relationships between Perceived Enjoyment and Perceived Ease of Use Heshan Sun Syracuse University hesun@syr.edu Ping Zhang Syracuse University pzhang@syr.edu ABSTRACT Causality
More informationFactorial Validity and Consistency of the MBI-GS Across Occupational Groups in Norway
Brief Report Factorial Validity and Consistency of the MBI-GS Across Occupational Groups in Norway Astrid M. Richardsen Norwegian School of Management Monica Martinussen University of Tromsø The present
More information1 7/17/2011 Measuring response distortion. Measuring response distortion using structural equation models. Michael D. Biderman
7/7/20 Measuring response distortion Measuring response distortion using structural equation models. Michael D. Biderman University of Tennessee at Chattanooga Department of Psychology 65 McCallie Ave.,
More informationValue Differences Between Scientists and Practitioners: A Survey of SIOP Members
Value Differences Between Scientists and Practitioners: A Survey of SIOP Members Margaret E. Brooks, Eyal Grauer, Erin E. Thornbury, and Scott Highhouse Bowling Green State University The scientist-practitioner
More informationThe Difference Analysis between Demographic Variables and Personal Attributes The Case of Internal Auditors in Taiwan
The Difference Analysis between Demographic Variables and Personal Attributes The Case of Internal Auditors in Taiwan Li-Jia Chiu and Neng-Tang Norman Huang Department of Technology Application and Human
More informationThe Doctrine of Traits. Lecture 29
The Doctrine of Traits Lecture 29 1 The Doctrine of Traits Allport (1937) [A trait is] a generalized and focalized neuropsychic system... with the capacity to render many stimuli functionally equivalent,
More informationPersonality: Definitions
Personality: Definitions Anastasi "Personality tests are instruments for the measurement of emotional, motivational, interpersonal and attitudinal characteristics, as distinct from abilities. Kaplan &
More information36th International Congress on Assessment Center Methods
36th International Congress on Assessment Center Methods A Different Look at Why Selection Procedures Work: The Role of Candidates Ability to Identify Criteria Martin Kleinmann University of Zurich, Switzerland
More informationTrue Colors Intl. True Colors Intl.
INTL. True True Colors Colors Intl. Intl. 1401 1401 N. Batavia N. Batavia St., St., Suite Suite 106 106 Orange, Orange, CA CA 92867 92867 800.422.4686 800.422.4686 www.truecolorsintl.com ww.truecolorsintl.com
More informationTo conclude, a theory of error must be a theory of the interaction between human performance variability and the situational constraints.
The organisers have provided us with a both stimulating and irritating list of questions relating to the topic of the conference: Human Error. My first intention was to try to answer the questions one
More informationTest Partnership TPAQ Series Psychometric Properties
Test Partnership TPAQ Series Psychometric Properties 2018 1 Construct Validity The IPIP-NEO-120 (Johnson, 2014) is a validated measure of the Big-5 model of personality, specifically the OCEAN model (Costa
More informationDOES SELF-EMPLOYED WORK MAKE 15-YEAR LONGITUDINAL PERSONALITY- BASED ANALYSIS
Frontiers of Entrepreneurship Research Volume 35 Issue 3 CHAPTER III. THE ENTREPRENEUR AND CHARACTERISTICS Article 2 6-13-2015 DOES SELF-EMPLOYED WORK MAKE INDIVIDUALS NOT ONLY MORE ENTREPRENEURIAL BUT
More informationPhysical Fitness Specialist Program Program Review October 24, 2017
Physical Fitness Specialist Program Program Review October 24, 2017 Catalog 2017-18 Program Description The Associate in Applied Science Degree in Physical Fitness Specialist is designed to meet the rising
More informationThe Innovate with C.A.R.E. Profile Research Report
The Innovate with C.A.R.E. Profile Research Report The Innovate with C.A.R.E. Research Report Item Number: O-063 1995 by Inscape Publishing, Inc. All rights reserved. Copyright secured in the US and foreign
More informationExtreme Response Style and Faking: Two Sides of the Same Coin?
Abstract Extreme Response Style and Faking: Two Sides of the Same Coin? Matthias Ziegler and Christoph J. Kemper Using Mixed Rasch Models, several studies identified two distinct classes present in personality
More informationPSYCHOLOGICAL PREDICTORS OF SOCIAL DESIRABILITY IN PERSONNEL SELECTION. Angana Bhattacharyya*, A. Suresh**& Joan Tirzah Selvaraj***
PSYCHOLOGICAL PREDICTORS OF SOCIAL DESIRABILITY IN PERSONNEL SELECTION Angana Bhattacharyya*, A. Suresh**& Joan Tirzah Selvaraj*** *, ***Junior Research Fellow, Defence Institute of Psychological Research,
More informationGroup Assignment #1: Concept Explication. For each concept, ask and answer the questions before your literature search.
Group Assignment #1: Concept Explication 1. Preliminary identification of the concept. Identify and name each concept your group is interested in examining. Questions to asked and answered: Is each concept
More informationEstimating Reliability in Primary Research. Michael T. Brannick. University of South Florida
Estimating Reliability 1 Running Head: ESTIMATING RELIABILITY Estimating Reliability in Primary Research Michael T. Brannick University of South Florida Paper presented in S. Morris (Chair) Advances in
More informationUsing self-report questionnaires in OB research: a comment on the use of a controversial method
JOURNAL OF ORGANIZATIONAL BEHAVIOR, VOL. 15,385-392 (1994) Using self-report questionnaires in OB research: a comment on the use of a controversial method PAUL E. SPECTOR University of South Florida, U.S.A.
More informationDAT Next Generation. FAQs
DAT Next Generation FAQs DAT TM Next Generation Frequently Asked Questions What does DAT Next Generation measure? The Differential Aptitude Tests, or DAT for short, are a battery of tests designed to assess
More informationToward an Understanding of Situational Judgment Item Validity and Group Differences
Journal of Applied Psychology 2011 American Psychological Association 2011, Vol. 96, No. 2, 327 336 0021-9010/11/$12.00 DOI: 137/a0021983 Toward an Understanding of Situational Judgment Item Validity and
More informationNeurotic Styles and the Five Factor Model of Personality
Graduate Faculty Psychology Bulletin Volume 3, No. 1, 2005 Neurotic Styles and the Five Factor Model of Personality Brian Norensberg, M.A. 1 & Peter Zachar Ph.D. 2 Abstract ~ This study investigates the
More informationLesson 12. Understanding and Managing Individual Behavior
Lesson 12 Understanding and Managing Individual Behavior Learning Objectives 1. Identify the focus and goals of individual behavior within organizations. 2. Explain the role that attitudes play in job
More informationThe happy personality: Mediational role of trait emotional intelligence
Personality and Individual Differences 42 (2007) 1633 1639 www.elsevier.com/locate/paid Short Communication The happy personality: Mediational role of trait emotional intelligence Tomas Chamorro-Premuzic
More informationPLANNING THE RESEARCH PROJECT
Van Der Velde / Guide to Business Research Methods First Proof 6.11.2003 4:53pm page 1 Part I PLANNING THE RESEARCH PROJECT Van Der Velde / Guide to Business Research Methods First Proof 6.11.2003 4:53pm
More informationIntroduction to the HBDI and the Whole Brain Model. Technical Overview & Validity Evidence
Introduction to the HBDI and the Whole Brain Model Technical Overview & Validity Evidence UPDATED 2016 OVERVIEW The theory on which the Whole Brain Model and the Herrmann Brain Dominance Instrument (HBDI
More informationAbstract. In this paper, I will analyze three articles that review the impact on conflict on
The Positives & Negatives of Conflict 1 Author: Kristen Onkka Abstract In this paper, I will analyze three articles that review the impact on conflict on employees in the workplace. The first article reflects
More informationPrepared By: Dr. Vijay Kumar Lecturer Department of Psychology PGGCG-11, Chandigarh
Prepared By: Dr. Vijay Kumar Lecturer Department of Psychology PGGCG-11, Chandigarh The Scope of Psychology As psychology has evolved as a science, its fields of specialization have multiplied and its
More informationVARIABLES AND MEASUREMENT
ARTHUR SYC 204 (EXERIMENTAL SYCHOLOGY) 16A LECTURE NOTES [01/29/16] VARIABLES AND MEASUREMENT AGE 1 Topic #3 VARIABLES AND MEASUREMENT VARIABLES Some definitions of variables include the following: 1.
More informationAssessment Information Brief: REVELIAN EMOTIONAL INTELLIGENCE ASSESSMENT (MSCEIT)
Assessment Information Brief: REVELIAN EMOTIONAL INTELLIGENCE ASSESSMENT (MSCEIT) Prepared by: Revelian Psychology Team E: psych@revelian.com P: (AU) or +61 7 3552 www.revelian.com 1 www.revelian.com 2
More informationValidity. Ch. 5: Validity. Griggs v. Duke Power - 2. Griggs v. Duke Power (1971)
Ch. 5: Validity Validity History Griggs v. Duke Power Ricci vs. DeStefano Defining Validity Aspects of Validity Face Validity Content Validity Criterion Validity Construct Validity Reliability vs. Validity
More informationThe Influence of Test Characteristics on the Detection of Aberrant Response Patterns
The Influence of Test Characteristics on the Detection of Aberrant Response Patterns Steven P. Reise University of California, Riverside Allan M. Due University of Minnesota Statistical methods to assess
More informationDUI SERVICE PROVIDER ORIENTATION DAY 2 AFTERNOON: DUI ASSESSMENT TOOLS
DUI SERVICE PROVIDER ORIENTATION DAY 2 AFTERNOON: DUI ASSESSMENT TOOLS 2017-2018 IDHS DUI ORIENTATION ASSESSMENT TOOLS My Training Room Rules richard.krajewski@dupageco.org updated 10/2017 Assessment Tools
More information5 Individual Differences:
5 Individual Differences: Self-Concept, Personality & Emotions Chapter From Self-Concept to Self-Management Personality: Concepts and Controversy Emotions: An Emerging OB Topic Self-Esteem 5-3 Self-Esteem
More informationPersonality and Social Psychology
Scandinavian Journal of Psychology, 2015, 56, 582 591 DOI: 10.1111/sjop.12231 Personality and Social Psychology Correction for faking in self-report personality tests LENNART SJ OBERG Center for Media
More informationPsychological testing
What is a psychological test Psychological testing An evaluative device or procedure in which a sample of an examinee s behavior in a specified domain is obtained and subsequently evaluated and scored
More informationHuman Relations: Interpersonal, Job-Oriented Skills CHAPTER 2 UNDERSTANDING INDIVIDUAL DIFFERENCES
Human Relations: Interpersonal, Job-Oriented Skills CHAPTER 2 UNDERSTANDING INDIVIDUAL DIFFERENCES Understanding individual differences is a key contributor to developing effective interpersonal relationships
More informationDUI SERVICE PROVIDER ORIENTATION
DUI SERVICE PROVIDER ORIENTATION Illinois Department of Human Services Division of Alcoholism and Substance Abuse Provided by the Institute for Legal, Legislative & Policy Studies Center for State Policy
More informationWorkplace Personality Inventory II
FREQUENTLY ASKED QUESTIONS TM Workplace Personality Inventory II May, 2014 QUESTIONS ON WPI II FEATURES In June 2013, the WPI II was released with two norm groups, a revised Profile Report, and a new Development
More informationAPS Interest Group for Coaching Psychologists (QLD)
APS Interest Group for Coaching Psychologists (QLD) Enhancing Emotional Intelligence via Coaching: Evidence from the field Presented by Dr Benjamin Palmer Director of Research & Development, Genos Proudly
More informationDavid V. Day John P. Hausknecht
Nonlinearity in Personality-Performance Relationships: An Examination of Source Effects David V. Day John P. Hausknecht Pennsylvania State University Paper presented at the 17 th Annual Conference of the
More informationExamining the Psychometric Properties of The McQuaig Occupational Test
Examining the Psychometric Properties of The McQuaig Occupational Test Prepared for: The McQuaig Institute of Executive Development Ltd., Toronto, Canada Prepared by: Henryk Krajewski, Ph.D., Senior Consultant,
More informationKCP learning. factsheet 44: mapp validity. Face and content validity. Concurrent and predictive validity. Cabin staff
mapp validity Face and content validity Careful design and qualitative screening of the questionnaire items in the development of MAPP has served to maximise both the face validity and the content validity
More informationMHR 405-Chapter 2. Motivation: The forces within a person that affect his or her direction, intensity and persistence of voluntary behaviour
MHR 405-Chapter 2 Motivation: The forces within a person that affect his or her direction, intensity and persistence of voluntary behaviour Figure 2.1 Ability: The natural aptitudes and learned capabilities
More informationEach copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission.
Decision-Making under Uncertainty in the Setting of Environmental Health Regulations Author(s): James M. Robins, Philip J. Landrigan, Thomas G. Robins, Lawrence J. Fine Source: Journal of Public Health
More informationTHE PROVISION OF HEARING AID SERVICES BY AUDIOLOGISTS
PRACTICE ADVICE THE PROVISION OF HEARING AID SERVICES BY AUDIOLOGISTS EFFECTIVE: FEBRUARY 2017 You may notice from the title that this document is different from our other PSGs. It contains only standards,
More informationThe construct validity of Emotional intelligence (EI) measurement among medical staffs of an emergency room in Taiwan
The construct validity of Emotional intelligence (EI) measurement among medical staffs of an emergency room in Taiwan Chia-Pang Shih, Kuang-Hung Hsu*, Department of Business Administration, Collage of
More informationIAASB Main Agenda (September 2005) Page Agenda Item. Analysis of ISA 330 and Mapping Document
IAASB Main Agenda (September 2005 Page 2005 1869 Agenda Item 4-D.1 Analysis of ISA 330 and Mapping Document 1. Exhibit 1 sets out statements in ISA 330 that use the present tense to describe auditor actions,
More information3rd International Conference on Science and Social Research (ICSSR 2014)
3rd International Conference on Science and Social Research (ICSSR 2014) Experimental Methods of Materiality Judgment on Auditor s Experience and Performance 1, 2, a Min Mao 1 Industrial and commercial
More information5/6/2008. Psy 427 Cal State Northridge Andrew Ainsworth PhD
Psy 427 Cal State Northridge Andrew Ainsworth PhD Some Definitions Personality the relatively stable and distinctive patterns of behavior that characterize an individual and his or her reactions to the
More informationMCORE TECHNICAL REPORT: DEVELOPMENT AND VALIDATION
MCORE TECHNICAL REPORT: DEVELOPMENT AND VALIDATION Todd W. Hall, Ph.D. Joshua Miller, Ph.D. Peter Larson, Ph.D. Pruvio, Inc. August, 2015 1 Copyright Standards This report contains proprietary research,
More informationINTERNATIONAL JOURNAL OF MANAGEMENT RESEARCH AND REVIEW
INTERNATIONAL JOURNAL OF MANAGEMENT RESEARCH AND REVIEW BIG FIVE INVENTORY (BFI) OF PERSONALITY TRAITS OF ENTREPRENEURS S.Subramanian* 1, Dr.V.Gopalakrishnan 2, Dr.I.Prem Rose Thayammal 3 1 Assistant Professor,
More informationReliability Theory for Total Test Scores. Measurement Methods Lecture 7 2/27/2007
Reliability Theory for Total Test Scores Measurement Methods Lecture 7 2/27/2007 Today s Class Reliability theory True score model Applications of the model Lecture 7 Psych 892 2 Great Moments in Measurement
More informationResponse to reviewer comment (Rev. 2):
Response to reviewer comment (Rev. 2): The revised paper contains changes according to comments of all of the three reviewers. The abstract was revised according to the remarks of the three reviewers.
More information