Statistical Analysis with Missing Data. Second Edition

Transcription

1 Statistical Analysis with Missing Data Second Edition

2 WILEY SERIES IN PROBABILITY AND STATISTICS Established by WALTER A. SHEWHART and SAMUEL S. WILKS Editors: David J. Balding, Peter Bloomfield, Noel A. C. Cressie, Nicholas I. Fisher, Iain M. Johnstone, J. B. Kadane, Louis M. Ryan, David W. Scott, Adrian F. M. Smith, Jozef L. Teugels Editors Emeriti: Vic Barnet, J. Stuart Hunder, David G. Kendall A complete list of the titles in this series appears at the end of this volume.

3 Statistical Analysis with Missing Data Second Edition RODERICK J. A. LITTLE DONALD B. RUBIN A JOHN WILEY & SONS, INC., PUBLICATION

4 Copyright # 2002 by John Wiley & Sons, Inc. All rights reserved. Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, , fax , or on the web at Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) , fax (201) , permcoordinator@wiley.com. Limit of Liability=Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of marchantability or fitness for a aprticular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other danages. For general information on our other products and services please contact our Customer Care Department within the U.S. at , outside the U.S. at or fax Wiley also publishes its books in a variety of electronic formats. Some content that appears in print, however, may not be available in electronic format. Library of Congress Cataloging-in-Publication Data Little, Roderick J. A. Statistical analysis with missing data = Roderick J Little, Donald B. Rubin nd ed. p. cm. - - (Wiley series in probability and statistics) A Wiley-Interscience publication. Includes bibliographical references and index. ISBN (acid-free paper) 1. Mathematical statistics. 2. Missing observations (Statistics) I Rubin, Donald B. II. Title. III. Series QA276.L dc ISBN Printed in the United States of America

5 Contents Preface xiii PART I OVERVIEW AND BASIC APPROACHES 1. Introduction The Problem of Missing Data, Missing-Data Patterns, Mechanisms That Lead to Missing Data, A Taxonomy of Missing-Data Methods, Missing Data in Experiments Introduction, The Exact Least Squares Solution with Complete Data, The Correct Least Squares Analysis with Missing Data, Filling in Least Squares Estimates, Yates s Method, Using a Formula for the Missing Values, Iterating to Find the Missing Values, ANCOVA with Missing-Value Covariates, Bartlett s ANCOVA Method, Useful Properties of Bartlett s Method, Notation, The ANCOVA Estimates of Parameters and Missing Y Values, ANCOVA Estimates of the Residual Sums of Squares and the Covariance Matrix of ^b, 31 v

6 vi CONTENTS 2.6. Least Squares Estimates of Missing Values by ANCOVA Using Only Complete-Data Methods, Correct Least Squares Estimates of Standard Errors and One Degree of Freedom Sums of Squares, Correct Least Squares Sums of Squares with More Than One Degree of Freedom, Complete-Case and Available-Case Analysis, Including Weighting Methods Introduction, Complete-Case Analysis, Weighted Complete-Case Analysis, Weighting Adjustments, Added Variance from Nonresponse Weighting, Post-Stratification and Raking To Known Margins, Inference from Weighted Data, Summary of Weighting Methods, Available-Case Analysis, Single Imputation Methods Introduction, Imputing Means from a Predictive Distribution, Unconditional Mean Imputation, Conditional Mean Imputation, Imputing Draws from a Predictive Distribution, Draws Based on Explicit Models, Draws Based on Implicit Models, Conclusions, Estimation of Imputation Uncertainty Introduction, Imputation Methods that Provide Valid Standard Errors from a Single Filled-in Data Set, Standard Errors for Imputed Data by Resampling, Bootstrap Standard Errors, Jackknife Standard Errors, Introduction to Multiple Imputation, Comparison of Resampling Methods and Multiple Imputation, 89

7 CONTENTS vii PART II LIKELIHOOD-BASED APPROACHES TO THE ANALYSIS OF MISSING DATA 6. Theory of Inference Based on the Likelihood Function Review of Likelihood-Based Estimation for Complete Data, Maximum Likelihood Estimation, Rudiments of Bayes Estimation, Large-Sample Maximum Likelihood and Bayes Inference, Bayes Inference Based on the Full Posterior Distribution, Simulating Draws from Posterior Distributions, Likelihood-Based Inference with Incomplete Data, A Generally Flawed Alternative to Maximum Likelihood: Maximizing Over the Parameters and the Missing Data, The Method, Background, Examples, Likelihood Theory for Coarsened Data, Factored Likelihood Methods, Ignoring the Missing-Data Mechanism Introduction, Bivariate Normal Data with One Variable Subject to Nonresponse: ML Estimation, ML Estimates, Large-Sample Covariance Matrix, Bivariate Normal Monotone Data: Small-Sample Inference, Monotone Data With More Than Two Variables, Multivariate Data With One Normal Variable Subject to Nonresponse, Factorization of the Likelihood for a General Monotone Pattern, Computation for Monotone Normal Data via the Sweep Operator, Bayes Computation for Monotone Normal Data via the Sweep Operator, Factorizations for Special Nonmonotone Patterns, 156

8 viii CONTENTS 8. Maximum Likelihood for General Patterns of Missing Data: Introduction and Theory with Ignorable Nonresponse Alternative Computational Strategies, Introduction to the EM Algorithm, The E and M Steps of EM, Theory of the EM Algorithm, Convergence Properties, EM for Exponential Families, Rate of Convergence of EM, Extensions of EM, ECM Algorithm, ECME and AECM Algorithms, PX-EM Algorithm, Hybrid Maximization Methods, Large-Sample Inference Based on Maximum Likelihood Estimates Standard Errors Based on the Information Matrix, Standard Errors via Methods that do not Require Computing and Inverting an Estimate of the Observed Information Matrix, Supplemental EM Algorithm, Bootstrapping the Observed Data, Other Large Sample Methods, Posterior Standard Errors from Bayesian Methods, Bayes and Multiple Imputation Bayesian Iterative Simulation Methods, Data Augmentation, The Gibbs Sampler, Assessing Convergence of Iterative Simulations, Some Other Simulation Methods, Multiple Imputation, Large-Sample Bayesian Approximation of the Posterior Mean and Variance Based on a Small Number of Draws, Approximations Using Test Statistics, Other Methods for Creating Multiple Imputations, 214

9 CONTENTS PART III LIKELIHOOD-BASED APPROACHES TO THE ANALYSIS OF INCOMPLETE DATA: SOME EXAMPLES ix 11. Multivariate Normal Examples, Ignoring the Missing-Data Mechanism Introduction, Inference for a Mean Vector and Covariance Matrix with Missing Data Under Normality, The EM Algorithm for Incomplete Multivariate Normal Samples, Estimated Asymptotic Covariance Matrix of ðy ^yþ, Bayes Inference for the Normal Model via Data Augmentation, Estimation with a Restricted Covariance Matrix, Multiple Linear Regression, Linear Regression with Missing Values Confined to the Dependent Variable, More General Linear Regression Problems with Missing Data, A General Repeated-Measures Model with Missing Data, Time Series Models, Introduction, Autoregressive Models for Univariate Time Series with Missing Values, Kalman Filter Models, Robust Estimation Introduction, Robust Estimation for a Univariate Sample, Robust Estimation of the Mean and Covariance Matrix, Multivariate Complete Data, Robust Estimation of the Mean and Covariance Matrix from Data with Missing Values, Adaptive Robust Multivariate Estimation, Bayes Inferences for the t Model, Further Extensions of the t Model, Models for Partially Classified Contingency Tables, Ignoring the Missing-Data Mechanism Introduction, 266

10 x CONTENTS Factored Likelihoods for Monotone Multinomial Data, Introduction, ML Estimation for Monotone Patterns, Precision of Estimation, ML and Bayes Estimation for Multinomial Samples with General Patterns of Missing Data, Loglinear Models for Partially Classified Contingency Tables, The Complete-Data Case, Loglinear Models for Partially Classified Tables, Goodness-of-Fit Tests for Partially Classified Data, Mixed Normal and Non-normal Data with Missing Values, Ignoring the Missing-Data Mechanism Introduction, The General Location Model, The Complete-Data Model and Parameter Estimates, ML Estimation with Missing Values, Details of the E Step Calculations, Bayes Computations for the Unrestricted General Location Model, The General Location Model with Parameter Constraints, Introduction, Restricted Models for the Cell Means, Loglinear Models for the Cell Probabilities, Modifications to the Algorithms of Sections and for Parameter Restrictions, Simplifications when the Categorical Variables are More Observed than the Continuous Variables, Regression Problems Involving Mixtures of Continuous and Categorical Variables, Normal Linear Regression with Missing Continuous or Categorical Covariates, Logistic Regression with Missing Continuous or Categorical Covariates, Further Extensions of the General Location Model, Nonignorable Missing-Data Models Introduction, 312

11 CONTENTS xi Likelihood Theory for Nonignorable Models, Models with Known Nonignorable Missing-Data Mechanisms: Grouped and Rounded Data, Normal Selection Models, Normal Pattern-Mixture Models, Univariate Normal Pattern-Mixture Models, Bivariate Normal Pattern-Mixture Models Identified via Parameter Restrictions, Nonignorable Models for Normal Repeated-Measures Data, Nonignorable Models for Categorical Data, 340 References 349 Author Index 365 Subject Index 371

12 Preface The literature on the statistical analysis of data with missing values has flourished since the early 1970s, spurred by advances in computer technology that made previously laborious numerical calculations a simple matter. This book aims to survey current methodology for handling missing-data problems and present a likelihood-based theory for analysis with missing data that systematizes these methods and provides a basis for future advances. Part I of the book discusses historical approaches to missing-value problems in three important areas of statistics: analysis of variance of planned experiments, survey sampling, and multivariate analysis. These methods, although not without value, tend to have an ad hoc character, often being solutions worked out by practitioners with limited research into theoretical properties. Part II presents a systematic approach to the analysis of data with missing values, where inferences are based on likelihoods derived from formal statistical models for the data-generating and missing-data mechanisms. Part III presents applications of the approach in a variety of contexts, including ones involving regression, factor analysis, contingency table analysis, time series, and sample survey inference. Many of the historical methods in Part I can be derived as examples (or approximations) of this likelihood-based approach. The book is intended for the applied statistician and hence emphasizes examples over the precise statement of regularity conditions or proofs of theorems. Nevertheless, readers are expected to be familiar with basic principles of inference based on likelihoods, briefly reviewed in Section 6.1. The book also assumes an understanding of standard models of complete-data analysis the normal linear model, multinomial models for counted data and the properties of standard statistical distributions, especially the multivariate normal distribution. Some chapters assume familiarity in particular areas of statistical activity analysis of variance for experimental designs (Chapter 2), survey sampling (Chapters 3, 4, and 5), or loglinear models for contingency tables (Chapter 13). Specific examples also introduce other statistical topics, such as factor analysis or time series (Chapter 11). The discussion of these examples is self-contained and does not require specialized knowledge, but such knowledge will, of course, enhance the reader s appreciation xiii

13 xiv PREFACE of the main statistical issues. We have managed to cover about three-quarters of the material in the book in a 40-hour graduate statistics course. When the first edition of this book was written in the mid-l980s, a weakness in the literature was that missing-data methods were mainly confined to the derivation of point estimates of parameters and approximate standard errors, with interval estimation and testing based on large-sample theory. Since that time, Bayesian methods for simulating posterior distributions have received extensive development, and these developments are reflected in the second edition. The closely related technique of multiple imputation also receives greater emphasis than in the first edition, in recognition of its increasing role in the theory and practice of handling missing data, including commercial software. The first part of the book has been reorganized to improve the flow of the material. Part II includes extensions of the EM algorithm, not available at the time of the first edition, and more Bayesian theory and computation, which have become standard tools in many areas of statistics. Applications of the likelihood approach have been assembled in a new Part III. Work on diagnostic tests of model assumptions when data are incomplete remains somewhat sketchy. Because the second edition has some major additions and revisions, we provide a map showing where to locate the material originally appearing in Edition 1. First Edition 1. Introduction 2. Missing Data in Experiments 3.2. Complete-Case Analysis 3.3. Available-Case Analysis 3.4. Filling in the Missing Values 4.2., 4.3. Randomization Inference with and without Missing Data 4.4. Weighting Methods 4.5. Imputation Procedures 4.6. Estimation of Sampling Variance with Nonresponse 5. Theory of Inference Based on the Likelihood Function 6. Factored Likelihood Methods 7. Maximum Likelihood for General Patterns of Missing Data Second Edition 1. Introduction 2. Missing Data in Experiments 3.2. Complete-Case Analysis 3.4. Available-Case Analysis 4.2. Imputing Means from a Predictive Distribution Omitted 3.3. Weighted Complete-Case Analysis 4. Imputation 5. Estimation of Imputation Uncertainty 6. Theory of Inference Based on the Likelihood Function 7. Factored Likelihood Methods, Ignoring the Missing-Data Mechanism 8. Maximum Likelihood for General Patterns 9.1. Standard Errors Based on the Information Matrix.

14 PREFACE First Edition 8. ML for Normal Examples 9. Partially Classified Contingency Tables 10.2 The General Location Model 10.3., Extensions Robust Estimation 11. Nonignorable Models 12.1., Survey Nonresponse Ignorable Nonresponse Models Multiple Imputation Nonignorable Nonresponse xv Second Edition 11. Multivariate Normal Examples 13. Partially Classified Contingency Tables 14. Mixed Normal and Categorical Data with Missing Values 12. Models for Robust Estimation 15. Nonignorable Models 3.3. Weighted Complete-Case Analysis 4. Imputation 5.4. Introduction to Multiple Imputation 10. Bayes and Multiple Imputation Normal Pattern-Mixture Models The statistical literature on missing data has expanded greatly since the first edition, in terms of scope of applications and methodological developments. Thus, we have not found it possible to survey all the statistical work and still keep the book of tolerable length. We have tended to confine discussion to applications in our own range of experience, and we have focused methodologically on Bayesian and likelihood-based methods, which we believe provide a strong theoretical foundation for applications. We leave it to others to describe other approaches, such as that based on generalized estimating equations. Many individuals are due thanks for their help in producing this book. NSF and NIMH (through grants NSF-SES , NSF-SES , NIMH-MH , DMS , and NSF ) helped support some aspects of the research reported here. For the first edition, Mark Schluchter helped with computations, Leisa Weld and T. E. Raghunathan carefully read the final manuscript and made helpful suggestions, and our students in Biomathematics M232 at UCLA and Statistics 220r at Harvard University also made helpful suggestions. Judy Siesen typed and retyped our many drafts, and Bea Shube provided kind support and encouragement. For the second edition, we particularly thank Chuanhai Liu for help with computation, and Mingyao Li, Fang Liu, and Ying Yuan for help with examples. Many readers have helped by finding typographical and other errors, and we particularly thank Adi Andrei, Samantha Cook, Shane Jensen, Elizabeth Stuart, and Daohai Yu for their help on this aspect. In closing, we continue to find that many statistical problems can be usefully viewed as missing-value problems even when the data set is fully recorded, and moreover, that missing-data research can be an excellent springboard for learning about statistics in general. We hope our readers will agree with us and find the book stimulating. Ann Arbor, Michigan Cambridge,Massachusetts R. J. A. LITTLE D. B. RUBIN