Computer Age Statistical Inference. Algorithms, Evidence, and Data Science. BRADLEY EFRON Stanford University, California

Transcription

1 Computer Age Statistical Inference Algorithms, Evidence, and Data Science BRADLEY EFRON Stanford University, California TREVOR HASTIE Stanford University, California ggf CAMBRIDGE UNIVERSITY PRESS

2 Preface Acknowledgments Notation xv xviii xix Part I Classic Statistical Inference 1 1 Algorithms and Inference A Regression Example Hypothesis Testing Notes 11 2 Frequentist Inference Frequentism in Practice Frequentist Optimality Notes and Details 20 3 Bayesian Inference TwoExamples Uninformative Prior Distributions Flaws in Frequentist Inference A Bayesian/Frequentist Comparison List Notes and Details 36 4 Fisherian Inference and Maximum Likelihood Estimation Likelihood and Maximum Likelihood Fisher Information and the MLE Conditional Inference Permutation and Randomization Notes and Details 51 5 Parametric Models and Exponential Families 53 ix

3 X 5.1 Univariate Families The Multivariate Normal Distribution Fisher's Information Bound for Multiparameter Families The Multinomial Distribution Exponent!al Families Notes and Details 69 Part II Early Computer Age Methods 73 6 Empirical Bayes Robbins' Formula The Missing-Species Problem A Medical Example Indirect Evidence Notes and Details 88 7 James-Stein Estimation and Ridge Regression The James-Stein Estimator The Baseball Players Ridge Regression Indirect Evidence Notes and Details Generalized Linear Models and Regression Trees Logistic Regression Generalized Linear Models Poisson Regression Regression Trees Notes and Details Survival Analysis and the EM Algorithm Life Tables and Hazard Rates Censored Data and the Kaplan-Meier Estimate The Log-Rank Test The Proportional Hazards Model Missing Data and the EM Algorithm Notes and Details The Jackknife and the Bootstrap The Jackknife Estimate of Standard Error The Nonparametric Bootstrap Resampling Plans 162

4 xi 10.4 The Parametric Bootstrap Influence Functions and Robust Estimation Notes and Details Bootstrap Confidence Intervals Neyman's Constraction for One-Parameter Problems The Percentile Method Bias-Corrected Confidence Intervals Second-Order Accuracy Bootstrap-? Intervals Objective Bayes Intervals and the Confidence Distribution Notes and Details Cross-Validation and C p Estimates of Prediction Error Prediction Rules Cross-Validation Covariance Penalties Training, Validation, and Ephemeral Predictors Notes and Details Objective Bayes Inference and MCMC Objective Prior Distributions Conjugate Prior Distributions Model Selection and the Bayesian Information Criterion Gibbs Sampling and MCMC Example: Modeling Population Admixture Notes and Details Postwar Statistical Inference and Methodology 264 Part III Twenty-First-Century Topics Large-Scale Hypothesis Testing and FDRs Large-Scale Testing False-Discovery Rates Empirical Bayes Large-Scale Testing Local False-Discovery Rates Choice of the Null Distribution Relevance Notes and Details Sparse Modeling and the Lasso 298

5 xii 16.1 Forward Stepwise Regression The Lasso Fitting Lasso Models Least-Angle Regression Fitting Generalized Lasso Models Post-Selection Inference for the Lasso Connections and Extensions Notes and Details Random Forests and Boosting Random Forests Boosting with Squared-Error Loss Gradient Boosting Adaboost: the Original Boosting Algorithm Connections and Extensions Notes and Details Neural Networks and Deep Learning Neural Networks and the Handwritten Digit Problem Fitting a Neural Network Autoencoders Deep Learning Learning a Deep Network Notes and Details Support-Vector Machines and Kernel Methods Optimal Separating Hyperplane Soft-Margin Classifier SVM Criterion as Loss Plus Penalty Computations and the Kernel Trick Function Fitting Using Kernels Example: String Kernels for Protein Classification SVMs: Concluding Remarks Kemel Smoothing and Local Regression Notes and Details Inference After Model Selection Simultaneous Confidence Intervals Accuracy After Model Selection Selection Bias Combined Bayes-Frequentist Estimation Notes and Details 4]7

6 xiii 21 Empirical Bayes Estimation Strategies Bayes Deconvolution g-modeling and Estimation Likelihood, Regularization, and Accuracy TwoExamples Generalized Linear Mixed Models Deconvolution and/-modeling Notes and Details 444 Epilogue 446 References 453 Author Index 463 Subject Index 467