Modern Regression Methods

Transcription

1 Modern Regression Methods Second Edition THOMAS P. RYAN Acworth, Georgia WILEY A JOHN WILEY & SONS, INC. PUBLICATION

2 Contents Preface 1. Introduction 1.1 Simple Linear Regression Model, Uses of Regression Models, Graph the Data!, Estimation of ßo and ß\, Orthogonal Regression, Inferences from Regression Equations, Predicting F, Worth of the Regression Equation, Regression Assumptions, Inferences on ß\, Inferences on ßo, Inferences for Y, Prediction Interval for Y, Confidence Interval for [iy\x, ANOVA Tables, Lack of Fit, Regression Through the Origin, Additional Examples, Correlation, Miscellaneous Uses of Regression, Regression for Control, Inverse Regression, Regression Control Chart, Monitoring Linear Profiles, 37

3 со 1.10 Fixed Versus Random Regressors, Missing Data, Spurious Relationships, Software, Summary, 40 Appendix, 41 References, 45 Exercises, 48 Diagnostics and Remedial Measures 2.1 Assumptions, Independence, Correlated Errors, AnExample, Corrective Action, Normality, Supernormality Property of Residuals, Standardized Deletion Residuals, Methods of Constructing Simulation Envelopes, Constant Variance, Weighted Least Squares, Unknown Weights, Modeling the Variance, A Heteroscedastic Alternative, Residual Plots, Transformations, Transforming the Model, Transforming the Regressors to Improve the Fit, Box-Tidwell Transformation, Transform Y to Obtain a Better Fit?, Transforming to Correct Heteroscedasticity and Nonnormality, Which R 2, Influential Observations, AnExample, Influence Statistics, Different Schools of Thought Regarding Influence, Modification of Standard Influence Measures, 105

4 vii Application of Influence Measures to Table 2.7 Data, Multiple Unusual Observations, Predicting Lifespan (?): An Influential Data Problem, Outliers, Measurement Error, Measurement Error in Y, Measurement Error in X, Software, Summary, 114 Appendix, 114 References, 116 Exercises, Regression with Matrix Algebra Introduction to Matrix Algebra, Eigenvalues and Eigenvectors, Matrix Algebra Applied to Regression, Predicted Y and R 2, Estimation of a 2, Variance of Y and f, Centered Data, Correlation Form, Influence Statistics in Matrix Form, Summary, 141 Appendix, 141 References, 142 Exercises, Introduction to Multiple Linear Regression An Example of Multiple Linear Regression, Orthogonal Regressors, Correlated Regressors, Partial-F Tests and f-tests, Individual Regressor Effects, Confidence Intervals and Prediction Intervals, Centering And Scaling, Centering, Scaling, 159

5 viii 4.3 Interpreting Multiple Regression Coefficients, MulticoUinearity and the "Wrong Signs" Problem, So Are Individual Regression Coefficients Interpretable?, Inflated Variances, Detecting MulticoUinearity, Variance Proportions, What to Do About MulticoUinearity?, Indicator Variables, Separation or Not?, Alternatives to Multiple Regression, Software, Summary, 177 References, 178 Exercises, Plots in Multiple Regression Beyond Standardized Residual Plots, Partial Residual Plots, CCPRPlot, Augmented Partial Residual Plots, CERES Plots, Some Examples, Which Plot?, Relationships Between Plots, True Model Contains Nonlinear Terms, Recommendations, Partial Regression Plots, Examples, Detrended Added Variable Plot, Partial Regression Plots Used to Detect Influential Observations, Other Plots For Detecting Influential Observations, Recent Contributions to Plots in Multiple Regression, Lurking Variables, Explanation of Two Data Sets Relative to R 2, Software, Summary, 227 References, 228 Exercises, 230

6 ix 6. Transformations in Multiple Regression Transforming Regressors, Transforming Y, Transformation Needed But Not Suggested, Transformation Needed and Suggested, Transformation Apparently Successful, Further Comments on the Normality Issue, Box-Сох Transformation, Box-Tidwell Revisited, Combined Box-Сох and Box-Tidwell Approach, Table 6.2 Data, Table 6.3 Data, Table 6.4 Data, MINITAB Tree Data, Other Analyses of the Tree Data, Stack Loss Data, Palm Beach County Data, Other Transformation Methods, Transform Both Sides (TBS), Transformation Diagnostics, Diagnostics After a Transformation, Software, Summary, 262 References, 262 Exercises, Selection of Regressors Forward Selection, Backward Elimination, Stepwise Regression, Significance Levels, All Possible Regressions, Criteria, Mallows's C p, Cp and Influential Data, Minimum a 2, t-statistics, Other Criteria, 277

7 X 7.5 Newer Methods, Robust Variable Selection, Examples, Variable Selection for Nonlinear Terms, Negative C p Values, Must We Use a Subset?, Model Validation, Software, Summary, 285 Appendix, 286 References, 287 Exercises, Polynomial and Trigonometric Terms Polynomial Terms, Orthogonal Polynomial Regression, When to Stop?, An Example, Polynomial-Trigonometric Regression, Orthogonality of Trigonometric Terms, Practical Considerations, Examples, Multiple Independent Variables, Software, Summary, 307 References, 308 Exercises, Logistic Regression Introduction, One Regressor, Estimating ß 0 and ßi, Method of Maximum Likelihood, Exact Logistic Regression, A Simulated Example, Complete and Quasicomplete Separation, Overlap: Modifying Table 9.1, Detecting Complete Separation, Quasicomplete Separation and Near Separation, 326

8 XI 9.5 Measuring the Worth of the Model, R 2 in Logistic Regression, Deviance, Other Measures of Model Fit, Determining the Worth of the Individual Regressors, Wald Test, Likelihood Ratio Test, Scores Test, Exact Conditional Scores Test, Exact p-value, Confidence Intervals, Confidence Interval for ßi, Confidence Interval for Change in Odds Ratio, Confidence Interval for тт, Exact Confidence Intervals, Exact Confidence Interval for ß x, Exact Confidence Interval for Change in Odds Ratio, Exact Prediction, Exact Confidence Interval for it, An Example With Real Data, Hosmer-Lemeshow Goodness-of-Fit Tests, Which Residuals?, Application to Table 9.4 Data, Pearson Residuals, Deviance Residuals, Other Diagnostics, Partial Residual Plot, Added Variable Plot, Confidence Intervals for Table 9.3 Data, An Example of Multiple Logistic Regression, Correct Classification Rate for Full Data Set, Influential Observations, Which Variables?, Algorithmic Approaches to Variable Selection, What About Nonlinear Terms?, Multicollinearity in Multiple Logistic Regression, 362

9 XÜ 9.12 Osteogenic Sarcoma Data Set, Missing Data, Sample Size Determination, Polytomous Logistic Regression, Logistic Regression Variations, Alternatives to Logistic Regression, Software for Logistic Regression, Summary, 375 Appendix, 375 References, 376 Exercises, Nonparametric Regression Relaxing Regression Assumptions, Bootstrapping, Monotone Regression, Smoothers, Running Line, Inferences for Running Line, Kernel Regression, Inferences in Kernel Regression, Local Regression, Inferences and Diagnostics, Splines, Piecewise Linear Regression (Linear Splines), Model Representation, Splines with Polynomial Terms, Smoothing Splines, Splines Compared to Local Regression, Other Smoothers, Which Smoother?, Smoothers for Multiple Regressors, Variable Selection, Important Considerations in Smoothing, Sliced Inverse Regression, Projection Pursuit Regression, Software, 411

10 xiü 10.9 Summary, 412 Appendix, 413 References, 414 Exercises, Robust Regression The Need for Robust Regression, Types of Outliers, Historical Development of Robust Regression, Breakdown Point, Efficiency, Classes of Estimators, M-Estimators, Bounded Influence Estimators, High Breakdown Point Estimators, Two-Stage Procedures, MM-Estimator (Three Stages), Goals of Robust Regression, Proposed High Breakdown Point Estimators, Least Median of Squares, Least Trimmed Squares, LTS Applications, S-Estimators, Approximating HBP Estimator Solutions, Application to Hawkins-Bradu-Kass Data Set, Another Application: One Regressor, A Proposed Sequential Procedure, Application to Multiple Regression, Other Methods for Detecting Multiple Outliers, Bounded Influence Estimators, Shortcomings of Bounded Influence Estimators, Application of Welsh Estimator, Multistage Procedures, Other Robust Regression Estimators, Applications, Software for Robust Regression, Summary, 457 References, 458 Exercises, 462

11 xiv 12. Ridge Regression Introduction, How Do We Determine kl, An Example, Ridge Regression for Prediction?, Generalized Ridge Regression, Inferences in Ridge Regression, Some Practical Considerations, Robust Ridge Regression?, Recent Developments in Ridge Regression, Other Biased Estimators, Software, Summary, 480 Appendix, 481 References, 482 Exercises, Nonlinear Regression Introduction, Linear Versus Nonlinear Regression, A Simple Nonlinear Example, Iterative Estimation, Relative Offset Convergence Criterion, Adequacy of the Estimation Approach, Computational Considerations, Determining Model Adequacy, Lack-of-Fit Test, Residual Plots, Multicollinearity Diagnostics, Influence and Unusual Data Diagnostics, Leverage, Influence, Inferences, Confidence Intervals, Prediction Interval, Hypothesis Tests, An Application, When Is a Linear Fit Not Good Enough?, 507

12 XV Rational Functions, Robust Nonlinear Regression, Applications, Teaching Tools, Recent Developments, Software, SAS Software, Cautions, SPSS, BMDP, S-Plus andtf, MINITAB, Summary, 513 Appendix, 513 References, 516 Exercises, Experimental Designs for Regression Objectives for Experimental Designs, Equal Leverage Points, Simple Linear Regression, Multiple Linear Regression, Construction of Equileverage Designs Two Regressors, Inverse Projection Approach, Other Desirable Properties of Experimental Designs, D-Optimality, G-Optimality, Other Optimality Criteria, Model Misspecification, Range of Regressors, Algorithms for Design Construction, Designs for Polynomial Regression, Designs for Logistic Regression, Designs for Nonlinear Regression, Software, Summary, 543 References, 544 Exercises, 547

13 xvi 15. Miscellaneous Topics in Regression Piecewise Regression and Alternatives, Semiparametric Regression, Quantile Regression, Poisson Regression, Exact Poisson Regression, Zero-Inflated Poisson Regression, Zero-Truncated Poisson Regression, Negative Binomial Regression, Zero-Inflated Negative Binomial Regression, Zero-Truncated Negative Binomial Regression, Cox Regression, Probit Regression, Censored Regression and Truncated Regression, Tobit Regression, Constrained Regression, Interval Regression, Random Coefficient Regression, Partial Least Squares Regression, Errors-in-Variables Regression, Regression with Life Data, Use of Regression in Survey Sampling, Bayesian Regression, Instrumental Variables Regression, Shrinkage Estimators, Meta-Regression, Classification and Regression Trees (CART), Multivariate Regression, 572 References, 572 Exercises, Analysis of Real Data Sets Analyzing Buchanan's Presidential Vote in Palm Beach County in 2000, Water Quality Data, Predicting Lifespan?, Scottish Hill Races Data, Leukemia Data, Y Binary, У Continuous, 598

14 r 16.6 Dosage Response Data, A Strategy for Analyzing Regression Data, Summary, 604 References, 604 Exercises, 606 XVH Answers to Selected Exercises 609 Statistical Tables 617 Author Index 625 Subject Index 637