Reliability and Validity in Neuropsychological Assessment Second Edition
CRITICAL ISSUES IN NEUROPSYCHOLOGY Series Editors Antonio E. Puente University of North Carolina at Wilmington Cecil R. Reynolds Texas A&M University and Bastrop Mental Health Associates Current Volumes in this Series CONTEMPORARY APPROACHES TO NEUROPSYCHOLOGICAL ASSESSMENT Edited by Gerald Goldstein and Theresa M. Incagnoli DETECTION OF MALINGERING DURING HEAD INJURY LmGATION Edited by Cecil R. Reynolds HANDBOOK OF CLINICAL CHll..D NEUROPSYCHOLOGY, Second Edition Edited by Cecil R. Reynolds and Elaine Fletcher-Janzen HANDBOOK OF CROSS-CULTURAL NEUROPSYCHOLOGY Edited by Elaine Fletcher-Janzen, Tony L. Strickland, and Cecil R. Reynolds HANDBOOK OF NEUROPSYCHOLOGY AND AGING Edited by Paul David Nussbaum INTERNATIONAL HANDBOOK OF NEUROPSYCHOLOGICAL REHABILITATION Edited by Anne-Lise Christensen and B. P. Uzzell MEDICAL NEUROPSYCHOLOGY, Second Edition Edited by Ralph E. Tarter, Meryl Butters, and Sue R. Beers NEUROPSYCHOLOGICAL INTERPRETATION OF OBJECTIVE PSYCHOLOGICAL TESTS Charles J. Golden, Patricia Espe-Pfeifer, and Jana Wachsler-Felder NEUROPSYCHOTHERAPY AND COMMUNITY INTEGRATION Brain lllness, Emotions, and Behavior Tedd Judd PRACTITIONER'S GUIDE TO EVALUATING CHANGE WITH INTELLECTUAL ASSESSMENT INSTRUMENTS Edited by Robert J. McCaffrey, Kevin Duff, and Holly J. Westervelt PRACTITIONER'S GUIDE TO EVALUATING CHANGE WITH NEUROPSYCHOLOGICAL ASSESSMENT INSTRUMENTS Edited by Robert J. McCaffrey, Kevin Duff, and Holly 1. Westervelt RELIABILITY AND VALIDITY IN NEUROPSYCHOLOGICAL ASSESSMENT, Second Edition Michael D. Franzen A Continuation Order Plan is available for this series. A continuation order will bring delivery of each new volume immediately upon publication. Volumes are billed only upon actual shipment. For further information please contact the publisher.
Reliability and Validity in Neuropsychological Assessment Second Edition Michael D. Franzen Allegheny General Hospital and Medical College of PennsylvanialHahnemann University Pittsburgh. Pennsylvania Springer Science+Business Media, LLC
Library of Congress Cataloging-in-Publication Data Franzen, Michael D., 1954- Reliability and validity in neuropsychological assessmentlmichael D. Franzen-2nd ed. p. ; cm. - (Critical issues in neuropsychology) Includes bibliographical references and index. I. Neuropsychological tests-evaluation. 2. Neuropsychological tests-validity. I. TItle. II. Series. [DNLM: I. Neuropsychological Tests. 2. Neuropsychology-methods. 3. Psychometrics-methods. 4. Reproducibility of Results. 5. Sensitivity and Specificity. WL 141 F837r 2000] RC386.6.N48 F73 2000 152-dc21 00-028725 ISBN 978-1-4419-3341-6 ISBN 978-1-4757-3224-5 (ebook) DOl 10.1007/978-1-4757-3224-5 2002 Springer Science+Business Media New York Originally published by Kluwer Academic 1 Plenum Publishers, New York in 2002. Softcover reprint of the hardcover 2nd edition 2002 http://www.wkap.nu ill 9 8 7 654 3 2 1 A C.l.P. record for this book is available from the Library of Congress All rights reserved No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording, or otherwise, without written permission from the Publisher
Preface Much has happened since the publication of the first edition of this book. Little of what has happened could have been predicted. The one accurate prediction was that clinical neuropsychology would continue to grow in terms of methods and knowledge as well as in importance in health care overall. The impact of managed care was not in the crystal ball. The rise in interest in the detection of response bias or in premorbid estimates was not seen in 1989. Both of those activities were viewed as being secondary and subsidiary to the "actual" testing. I did not foresee the rise in interest and sophistication in psychometric issues among clinical neuropsychologists, although certainly I am gratified by this development. When I began my career, I had a great deal of difficulty trying to publish papers that were studies of reliability and validity, although certainly validity studies had a better shot at being accepted. Today, psychometrically based studies are routinely published in a newer journal, The Clinical Neuropsychologist, and to a somewhat lesser extent in other journals such as Archives of Clinical Neuropsychology, Journal of Experimental and Clinical Neuropsychology, and Psychological Assessment. Clinical neuropsychologists are more sophisticated in discussing issues of sensitivity, specificity, advanced concepts in reliability, and confirmatory factor analysis. Writing the second edition of this book was a much more labor-intensive effort than the first edition. The names of certain of my colleagues would pop up time after time in my reviews of the journals. In fact, during some late nights I would silently curse the productivity of some of these colleagues whose output was slowing the completion of the book. Psychometric research is not sexy, but it has the potential to inform our clinical practice in an enormous way. Therefore, these researchers deserve our thanks, and in the morning light I would retract my nighttime curses. The first edition of this book did not initiate this wave of psychometric sophistication, but it did ride the crest. Other books (e.g., Mitrushina, Boone, & D'Elia, 1999; Spreen & Strauss, 1998) address the issues of normative information from a different perspective. The intent of this book is to review the extensive research and hopefully provide a direction for additional investigation. To evaluate the degree to which the numbers produced by assessment possessed the characteristics of stability and meaningfulness, or of reliability and validity, experimental and mathematical methods were applied that had been developed in other areas of psychometrics (literally, the measurement of psychological functions). But because clinical neuropsychology still remained at least partly a medical endeavor, it was somewhat slower than v
Vi PREFACE some areas of psychological measurement to evaluate its instruments by psychometric methods. There were even some individuals who totally eschewed the use of assessment methods that produced numbers. That is not the case today. The practitioners who are most closely identified with psychometric methods are those clinical neuropsychologists who use a battery approach to assessment. But even those individuals who use a flexible approach, sometimes called a process approach, rely on smaller tests that produce numbers. It is difficult to find a contemporary clinical neuropsychologist who does not use at least a few standardized tests in evaluating a patient. The evaluation of the psychometric properties of clinical neuropsychological assessment instruments has assumed a more prominent position and a higher priority than it held previously. The psychometric evaluation of assessment instruments is a continual activity. This is true, first of all, because psychometric theoreticians keep producing innovations. Second, the psychometric properties of the instruments depend on the characteristics of the subject populations. As individuals change or as different emphases develop for a population, the instruments need to be reevaluated. This book should be seen as a takeoff rather than as a summation point. The purpose of this book is not to indict certain assessment instruments or to champion other instruments. Rather, the purpose is to identify areas of needed research. Recommendations regarding clinical use are made in certain cases; however, if we were to use only those instruments that have been completely evaluated and found to be completely satisfactory, our repertoire of assessment techniques would be very small indeed. This last statement is not meant to be a justification for the use of incompletely evaluated instruments or for the use of instruments that have been evaluated and found to lack reasonable reliability or to generate no valid interpretations. We have an obligation to choose the best instrument or to develop a better one. Some instruments may be useful in some situations and useless in others. We should use the results of research to guide our choice of assessment instruments. It is hoped that this book will alert the clinical researcher to the need to conduct basic research into the psychometric properties of neuropsychological instruments. It is further hoped that clinicians will use the book to make decisions regarding the use of instruments for particular patients in particular situations. It is also hoped that students will be able to use this book to learn about instruments or to generate ideas for thesis or dissertation research.
Acknowledgments I'd like to thank the coauthors of the first edition of this book, Robert Sawicki and Doug Robbins, who helped shape the initial manifestation. I'd like to acknowledge the enormous contributions of my colleagues in clinical neuropsychology, without whose work this book would not have been possible, or even necessary. I'd like to thank my editor, Mariclaire Cloutier, who showed great patience in awaiting the completion of the manuscript. Most of all I'd like to acknowledge Debbie, who showed great forbearance while I toiled, and Joe, Tim, and Rose who accepted Dad's preoccupation and who learned to write at their computers with professional journals propped open at the side. vii
Contents Chapter 1 Preliminary Measurement Considerations in Clinical Neuropsychological Assessment... 1 Chapter 2 General and Theoretical Considerations in the Assessment of Reliability 7 Chapter 3 Practical and Methodological Considerations Regarding Reliability 15 Chapter 4 Elemental Considerations in Validity 27 Chapter 5 Validity as Applied to Neuropsychological Assessment 33 Chapter 6 The Wechsler Adult Intelligence Scale-Revised and Wechsler Adult Intelligence Scale-III... 55 Chapter 7 The Wechsler Intelligence Scales for Children-the WISC-R, WISC-III, and WPPSI-R... 71 Chapter 8 Tests of General Intelligence 91 ix
x CONTENTS Chapter 9 The Halstead-Reitan and Luria-Nebraska Neuropsychological Batteries 115 Chapter 10 Benton's Neuropsychological Assessment.......................... 153 Chapter 11 The Minnesota Multiphasic Personality Inventory and Personality Assessment Inventory... 171 Chapter 12 The Rorschach Inkblots 193 Chapter 13 The Wechsler Memory Scale and its Revisions 203 Chapter 14 Tests of Memory 229 Chapter 15 Tests of Verbal Functions 255 Chapter 16 Tests of Visual and Construction Functions 267 Chapter 17 Tests of Higher Cognitive Function 283 Chapter 18 Screening Devices............................................ 299 Chapter 19 Tests of Achievement and Aptitude 319 Chapter 20 Methods for Evaluating the Validity of Test Scores 331
CONTENTS xi Chapter 21 The Assessment of Child Neuropsychological Function............... 349 Postscript-Future Directions... 355 References... 357 Index... 449