Progressive Matrices

Transcription

1 Seeing Reason: Visuospatial Ability, Sex Differences and the Raven s Progressive Matrices Nicolette Amanda Waschl School of Psychology, University of Adelaide A thesis submitted in fulfillment of the requirements for the degree of Doctor of Philosophy March, 2017

2

3 iii Table of Contents Table of Contents... iii List of Tables... ix List of Figures... xi List of Abbreviations... xiii Abstract... xvi Declaration... xviii Acknowledgements... xix Chapter 1: Literature Review and Introduction... 1 Intelligence Theory... 2 Cattell-Horn-Carroll Theory... 3 Other Theoretical Frameworks The Raven s Progressive Matrices Validity and Reliability of the RPM The Dimensionality Problem Preliminary Conclusions: The RPM, Dimensionality and Gv Sex Differences in Cognitive Abilities Methodological Issues in the Study of Sex Differences Sex Differences in Gv Sex Differences in Gf General Conclusions and Remaining Questions Chapter 2: Exegesis Preamble Issue One: The RPM Tests Structure of the RPM Tests... 52

4 iv Validity of the RPM Tests New Contributions Issue Two: Intelligence Theory The relationship between Gf and Gv New Contributions Issue Three: Sex Differences New Contributions Issues Beyond the Scope of this Thesis Chapter 3: Paper Preamble Rule Synthesis Results of Rule Synthesis Dillon et al. s (1981) Factor Analysis Preliminary Conclusions from Solution Taxonomy Synthesis Paper Abstract Method Participants and Measures Data Analysis Results Sex Differences in Items and Item Types Confirmatory Factor Analysis Measurement Invariance and Latent Mean Differences Rasch Analysis and Differential Item Functioning Discussion

5 v Chapter 4: Paper Preamble Paper Abstract Method Participants Measures Design Analysis Results Descriptive Statistics Structural Equation Models Sex Differences Discussion Chapter 5: Paper Methodology Selection of Measures to Include Literature Search Search Terms Sample Three Grey Literature Reference Lists of Identified Papers Data Collection Paper Abstract

6 vi Theoretical Framework Previous Meta-Analyses Moderators Present Study Method Selection of Measures to Include Literature Search Inclusion Criteria Meta-Analytic Techniques and Procedures Results Descriptive Statistics Heterogeneity and Subgroup Analyses Sensitivity Analysis and Publication Bias Discussion Item Stimuli and Item Type Individual Tests Limitations Implications for Future Research and Practice Chapter 6: Paper Preamble Paper Abstract Method Participants and Data Preparation Results

7 vii Exploratory Factor Analysis Exploratory Structural Equation Modeling (ESEM) Confirmatory Factor Analysis Multitrait-Multimethod Analysis Discussion Chapter 7: General Discussion and Research Conclusions Research Conclusions: Paper Research Conclusions: Paper Research Conclusions: Paper Research Conclusions: Paper General Discussion: Significance and Implications RPM as a measure of Gf Sex Differences, Gv and the RPM RPM and Academic Achievement Alternative Explanations Implications for Intelligence Theory and Measurement Measurement of Inductive Reasoning Sex Differences in Inductive Reasoning Test-specific versus Content-specific Variance Figural and Alphabetic Inductive Reasoning Developmental Differences Manifest and Latent Differences Strategy Differences General Discussion: Limitations and Future Research RPM and Gv

8 viii Flexibility of Closure Components of RPM Performance, Gv and Sex Differences Short Forms Sex Differences and Measuring Inductive Reasoning Sample Representativeness Letter Series Tests Experimental Investigations Strategy Use Final Comments References Appendix A Appendix B Appendix C Appendix D

9 ix List of Tables Table 1.1 Table 1.2 Table 1.3 Table 3.1 Description of the sixteen broad abilities under the CHC model... 5 Solution strategies from Carpenter et al. (1990) Solution strategies from DeShon et al. (1995) Comparison of Modified DeShon et al. (1995) with Carpenter et al. (1990) Table 3.2 Paper 1: Table 1 Paper 1: Table 2 Paper 1: Table 3 Paper 1: Table 4 Paper 1: Table 5 Paper 1: Table 6 Paper 1: Table 7 Paper 1: Table 8 Paper 1: Table 9 Paper 1: Table 10 Paper 1: Table 11 Paper 2: Table 1 Paper 2: Table 2 Paper 2: Table 3 Paper 2: Table 4 Paper 2: Table 5 Paper 2: Table 6 Dillon et al. s (1981) item types Classifications of APM Items Sex differences in individual items Sex differences in item types Model fit indices: Sample Model fit indices: Sample Model fit indices: Sample Measurement invariance statistics Item fit and DIF: Sample Misfitting items: Sample Item fit and DIF: Sample Item fit and DIF: Sample Descriptive Statistics Correlations: APM Sample Correlations: APM Sample 2 and SPM Sample Fit Statistics for Measurement Models: APM Sample Fit Statistics for Measurement Models: SPM Sample Measurement invariance statistics

10 x Table 5.1 Paper 3: Table 1 Years searched Measures of inductive reasoning identified and included in searches Paper 3: Table 2 Paper 3: Table 3 Paper 3: Table 4 Paper 4: Table 1 Studies included in analysis Number of data points for each test category Summary statistics by test category and test Subtest means for Group 1 and 2 compared to 2007 and 2012 Australian norms Paper 4: Table 2 Factor loadings and communalities for the two-factor solution (Gf/Gc) GRT2 test Paper 4: Table 3 Paper 4: Table 4 Paper 4: Table 5 Paper 4: Table 6 Verbal reasoning items by factor Average loadings of different item types on Factors I and II Items demonstrating Differential Item Functioning Factor loadings and threshold values for GRT2 two-factor confirmatory model Paper 4: Table 7 Table 7.1 Table 7.2 Table 7.3 Measurement invariance statistics Means and standard deviations Correlations Regression models

11 xi List of Figures Figure 3.1 a) Addition/Subtraction; b) Superimposition; c) Superimposition with cancellation Figure 3.2 Figure 3.3 Superimposition with conditional placement a) Quantitative pairwise progression; (b) Movement; (c) Rotation Paper 1: Figure 1 Test information curves for the combined group for each sample Paper 1: Figure 2 Paper 1: Figure 3 Paper 1: Figure 4 Paper 2: Figure 1 ICCs for three representative items from Sample ICCs for three representative items from Sample ICCs for three representative items from Sample APM Sample 1 Model: Structural equation model showing standardized coefficients for reasoning and rotation to latent APM and residual latent APM Paper 2: Figure 2 APM Sample 2 Model 1: Structural equation model showing standardized coefficients for latent Gv and inductive reasoning to latent APM and residual latent APM Paper 2: Figure 3 APM Sample 2 Model 2: Structural equation model showing standardized coefficients for ability measures to latent APM and residual latent APM Paper 2: Figure 4 SPM Model 1: Structural equation model showing standardized coefficients for latent Gv and inductive reasoning to latent SPM and residual latent SPM

12 xii Paper 2: Figure 5 SPM Model 2: Structural equation model showing standardized coefficients for ability measures to latent SPM and residual latent SPM Paper 3: Figure 1 Forest plot of effect sizes for the Advanced Progressive Matrices Paper 3: Figure 2 Forest plot of effect sizes for the Standard Progressive Matrices Paper 3: Figure 3 Paper 3: Figure 4 Forest plot of effect sizes for the WAIS Matrix Reasoning Forest plot of effect sizes for the Differential Aptitude Test Abstract Reasoning Paper 3: Figure 5 Forest plot of effect sizes for the Differential Aptitude Test Verbal Reasoning Paper 3: Figure 6 Forest plot of effect sizes for the Primary Mental Abilities - Reasoning Paper 3: Figure 7 Paper 3: Figure 8 Paper 3: Figure 9 Paper 4: Figure 1 Paper 4: Figure 2 Paper 4: Figure 3 Forest plot of effect sizes for the CFIT Forest plot of effect sizes for the WAIS - Similarities Funnel plot Factor scores by sex Factor scores by age Second-order correlated uniqueness model

13 xiii List of Abbreviations 2PL 3PL AH APM AR BIS CAB-I CAB-Cf CFA CFI CFIT CHC CPM CTT DAS DAT DAT-AR DAT-VR DIF DWLS EA EFA ESEM ETS Two-parameter logistic Three-parameter logistic Alice Heim Test Raven s Advanced Progressive Matrices Abstract Reasoning (GRT2 subtest) Berlin Structure of Intelligence Comprehensive Ability Battery Inductive Reasoning Comprehensive Ability Battery Flexibility of Closure Confirmatory Factor Analysis Comparative Fit Index Cattell s Culture Fair Intelligence Test Cattell-Horn-Carroll Raven s Coloured Progressive Matrices Classical Test Theory Differential Aptitude Scales Differential Aptitude Tests Differential Aptitude Tests Abstract Reasoning Differential Aptitude Tests Verbal Reasoning Differential item functioning Diagonally weighted least squares Esoteric Analogies Test Exploratory factor analysis Exploratory structural equation modeling Educational Testing Service

14 xiv g Gc Gf General intelligence Crystallised ability Fluid ability GRT2 General Reasoning Test 2 Gq Gv I ICC IQ IRT IST K-BIT K-SNAP KAIT MGCFA MIMIC ML MRT MTMM NR PCA PF PMA-R PSVT:R RG Quantitative ability Visuospatial ability Induction Item characteristic curve Intelligence Quotient Item Response Theory Intelligence Structure Test Kaufman Brief Intelligence Test Kaufman Short Neuropsychological Assessment Procedure Kaufman Adolescent and Adult Intelligence Test Multiple-group confirmatory factor analysis Multiple-indicator Multiple-causes Maximum likelihood Mental Rotation Test Multitrait-multimethod Numerical Reasoning (GRT2 subtest) Principal components analysis Paper Folding Test Primary Mental Abilities Reasoning Perdue Spatial Visualization Test of Rotations Sequential reasoning

15 xv RMSEA RPM RQ SEM SRMR SPM US UK VPR VR WAIS WLSMV WJ-III Root Mean Square Error of Approximation Raven s Progressive Matrices Quantitative reasoning Structural Equation Modeling Standardized Root Mean Square Residual Standard Raven s Progressive Matrices United States United Kingdom Verbal-Perceptual-Image Rotation Verbal Reasoning (GRT2 subtest) Wechsler Adult Intelligence Scale Weighted least squares mean and variance adjusted Woodcock-Johnson Test of Cognitive Abilities 3 rd Edition

16 xvi Abstract This thesis sought to address the role of visuospatial ability in measures of inductive reasoning, with a particular focus on the Raven s Progressive Matrices (RPM). Given that males tend to perform better on certain measures of visuospatial ability, sex differences in performance on the RPM tests and in other measures of inductive reasoning were also examined. The issue of the involvement of visuospatial ability in the RPM tests is important at both a practical and a theoretical level. At the practical level, these tests are often used as a sole measure of general intelligence, and conclusions regarding the relationship of general intelligence to other variables are made on the basis of results from this test. If the RPM tests require a substantive amount of visuospatial ability, this is problematic to the interpretation of results on this test as reflective of general intelligence. At a theoretical level, investigation of this question pertains to an understanding of the relationship between visuospatial abilities and fluid ability generally, but inductive reasoning more specifically. Many commonly used measures of inductive reasoning are presented in a visual format (e.g. abstract figures) and these tests are often shown to cross-load on both fluid and visuospatial factors. This thesis addresses the issues of visuospatial ability and sex differences in the RPM by examining (1) the dimensionality of the Advanced RPM tests; (2) the role of Gv in performance on the RPM tests; and (3) sex differences in raw scores on the RPM and other measures of inductive reasoning. Additionally, the psychometric properties of the General Reasoning Test 2 (GRT2) in the Australian population were examined. This included an investigation of the relationship between figural, verbal and numeric reasoning items as well as sex differences.

17 xvii Study 1 used confirmatory factor analysis and Rasch modeling to investigate the dimensionality of the Advanced RPM, measurement invariance and differential item functioning across sex. Study 2 used structural equation modeling to examine, in three separate samples, how well visuospatial abilities could account for the variance in a latent RPM factor not already accounted for by alternative fluid ability measures. This study additionally assessed invariance of the structural relationships between visuospatial ability, fluid ability and RPM across sex. Study 3 used meta-analytic techniques to synthesise research concerning sex differences on measures of inductive reasoning, considering the item stimulus and item type as potential moderators of this difference. Study 4 used exploratory and confirmatory structural equation modeling to examine the psychometric properties of the GRT2. Results indicate that although the RPM tests are largely unidimensional, visuospatial ability is involved in performance. Furthermore, sex differences in raw scores and at the latent level were found, favouring males. Investigation of sex differences in inductive reasoning measures more broadly indicated that the figural format of these tests may contribute to the male advantage often identified; however, examination of the influence of the stimulus and type of question used in reasoning items in the GRT2 indicated that these do not meaningfully impact the latent construct measured.

18 xviii Declaration I certify that this work contains no material which has been accepted for the award of any other degree or diploma in my name, in any university or other tertiary institution and, to the best of my knowledge and belief, contains no material previously published or written by another person, except where due reference has been made in the text. In addition, I certify that no part of this work will, in the future, be used in a submission in my name, for any other degree or diploma in any university or other tertiary institution without the prior approval of the University of Adelaide and where applicable, any partner institution responsible for the joint-award of this degree. I give consent to this copy of my thesis when deposited in the University Library, being made available for loan and photocopying, subject to the provisions of the Copyright Act I acknowledge that copyright of published works contained within this thesis resides with the copyright holder(s) of those works. I also give permission for the digital version of my thesis to be made available on the web, via the University s digital research repository, the Library Search and also through web search engines, unless permission has been granted by the University to restrict access for a period of time. I acknowledge the support I have received for my research through the provision of an Australian Government Research Training Program Scholarship. Nicolette Waschl Date: 29/03/17

19 xix Acknowledgements Firstly, I would like to express my sincere gratitude to my principal supervisor, Professor Nick Burns; thank you for you patience, kindness and knowledge. I am grateful to have had a supervisor whom I always felt had my best interests at heart and whom I knew I was always able to approach with any concerns or questions I may have had. Thank you also to my co-supervisors Emeritus Professor Ted Nettelbeck and Dr. Irina Baetu, for their support during my PhD study. To Ted for always taking an interest in my work and providing invaluable feedback, and to Irina for the programming lessons and helpful chats. My sincere thanks also goes to others who have enabled me to conduct this research; Dr. Simon Jackson, who provided data used in Papers 1 and 2, Stephen Kohl who provided data used in Paper 4, and Professor Andy Baker who was involved in providing data used in Paper 2. Also thank you to those authors who kindly provided me with unpublished information subsequently included in Paper 3. Thank you to Professor Jane Mathias and Associate Professor Siva Alagumalai for sharing with me their knowledge of meta-analysis and Rasch analysis. I would like to thank Daniel McCluskey for his support during my time spent as a PhD candidate; you have been endlessly patient and understanding with my never-ending student status. Thank you also to my parents who have supported and encouraged me always, and to mum for taking the time to proof this document. Finally, thank you to my fellow PhD students and friends, particularly Erica, Michael and Jen, for all the coffee and beer (and wine and cheese).