Chapter 11 Multiple Regression PSY 295 Oswald Outline The problem An example Compensatory and Noncompensatory Models More examples Multiple correlation Chapter 11 Multiple Regression 2 Cont. Outline--cont. The Problem Residuals Hypothesis testing Moderated multiple regression Short summary Review questions Using several predictors (IVs) to predict the dependent variable (DV) Finding a measure of overall model fit Weighting each predictor Chapter 11 Multiple Regression 3 Chapter 11 Multiple Regression 4 1
Multiple Regression (MR) We can use more than one predictor to predict a criterion 2 Predictors: ( 1 ) mechanical aptitude, ( 2 ) conscientiousness Criterion: (Y) job performance at Chevy (supervisory ratings) Chapter 11 Multiple Regression 5 Mechanical Aptitude and Conscientiousness Predicting Job Performance Y j o 6 b 5 p e 4 r f 3 o r 2 m a 1 n c 70 e 20 30 60 40 50 50 40 Chapter 11 Multiple Regression mechanical aptitude 6 2 1 conscientiousness 3-D data plot Mechanical Aptitude (MA) and Conscientiousness (CS) Predicting Job Performance (PF) The Math of MR PF Y 6 5 4 3 2 1 0 40 44 48 51 55 59 63 66 70 40 Chapter 11 Multiple Regression 7 MA 51 61 3-D regression plane CS Mathematically, we just add another regression weight and another variable to the regression equation Yˆ = a+ b + b 1 1 2 2 The goal of regression is the same as before but in 3-D: minimize the deviations around the regression plane (which maximizes prediction using the plane) Look! Oh my! In geekspeak: minimize the sum of (actual Y predicted Y) 2 Chapter 11 Multiple Regression 8 2
Implications of MR 1. Regression is very complex when predictors are correlated the plane is fit that predicts best, the weights do NOT imply which of the variables is more important Y$ = a + b + b 1 1 2 2 Chapter 11 Multiple Regression 9 More Implications of MR 2. Regression is compensatory Y $ =. 075+. 04 +. 025 1 2 Person 1 gets low mech = 35 and high CS = 50: Y ˆ =.075 +.04( 35) +.025( 50) = 2.58 Person 2 gets high mech = 51 and low CS = 25: Y ˆ =.075 +.04( 51) +.025( 25) = 2.59 Chapter 11 Multiple Regression 10 Noncompensatory Models (not MR) Many real-world situations are noncompensatory: Can t go to sleep until you ve finished your reading Licensure (can t practice until you have a license) Certain types of beverage purchases aren t allowed until a certain age Can t become an esteemed professor until you ve got yer PhD Chapter 11 Multiple Regression 11 Noncompensatory Models (not MR) Sometimes models are noncompensatory because it s too impractical otherwise (too cost-, time-, laborintensive) Chapter 11 Multiple Regression 12 3
Noncompensatory Models (not MR) Individuals applying to become an airline pilot first take an flight knowledge test and an interpersonal skills test You test those with high scores (top 10%) on an expensive flight simulator. Those with high simulator scores get hired. The selection model as a whole is noncompensatory: Simulator performance doesn t compensate for low knowledge and skills scores (as they were selected out). Chapter 11 Multiple Regression 13 Back to MR Another Example: Predicting Trait Anxiety We can use more than one predictor to predict a criterion we re interested in 2 Predictors: ( 1 ) birthweight, ( 2 ) ratings of maternal attachment Criterion: (Y) self-reported trait anxiety at age 12 Chapter 11 Multiple Regression 14 Back to MR Things to Know Multiple Predictors in Regression It is important to know C - AN The validities of the predictors (predictor-criterion correlations) The intercorrelations of the predictors (predictor-predictor correlations) P1 - BWT P2 - ATT Case 1: Low predictor intercorrelations Chapter 11 Multiple Regression 15 Chapter 11 Multiple Regression 16 4
Multiple Predictors in Regression C - AN refers to the amount that one predictor adds to the prediction of another P1 - BWT P2 - ATT Case 2: High predictor intercorrelations So in predicting anxiety: Case 1, the attachment over birthweight is high Case 2, the attachment is low of of Chapter 11 Multiple Regression 17 Chapter 11 Multiple Regression 18 Collecting Additional Predictor Information for Regression An Example The best additional predictors to add are ones that are (not correlated) with other predictor information already used in the MR (i.e., predictors with high ). Chapter 11 Multiple Regression 19 Study by Kliewer et al. (1998) on effect of violence on internalizing behavior Define internalizing behavior Predictors Degree of witnessing violence Measure of life stress Measure of social support Chapter 11 Multiple Regression 20 5
Violence and Internalizing Intercorrelation Matrix Subjects are children 8-12 years Lived in high-violence areas Hypothesis: violence and stress lead to internalizing behavior. Data available at www.uvm.edu/~dhowell/statpage s/more_stuff/kliewer.dat Pearson Correlation Amount violenced witnessed Current stress Social support Internalizing symptoms on CBCL Correlations Amount violenced witnessed Current stress.050.080 -.080 Social support.200*.270** -.170 *. Correlation is significant at the 0.05 level (2-tailed). **. Correlation is significant at the 0.01 level (2-tailed). Internalizin g symptoms on CBCL Chapter 11 Multiple Regression 21 Chapter 11 Multiple Regression 22 Preliminary Stuff Multiple Correlation Note that both Stress and Witnessing Violence are significantly correlated with Internalizing. Note that predictors are largely independent of each other. Directly analogous to simple r Always capitalized (e.g. R) Always Correlation of Ŷ predicted Y (Y-hat) with observed Y Ŷ where predicted Y (Y-hat) is computed from regression equation Often reported as Chapter 11 Multiple Regression 23 Chapter 11 Multiple Regression 24 6
Regression Coefficients Model Summary R R Square Adjusted R Square Std. Error of the Estimate.37 a.135.108 2.2198 a. Predictors: (Constant), Social support, Current stress, Amount violenced witnessed Slopes and an intercept. Each variable adjusted for all others in the model. Just an extension of slope and intercept in simple regression SPSS output on next slide Chapter 11 Multiple Regression 25 Chapter 11 Multiple Regression 26 Slopes and Intercept Regression Equation (Constant) Amount violenced witnessed Current stress Social support Coefficients a Unstandardized Coefficients Standardized Coefficients Std. B Error Beta t Sig..477 1.289.37.712.038.018.201 2.1.039.273.106.247 2.6.012 -.074.043 -.166-2.087 a. Dependent Variable: Internalizing symptoms on CBCL Yˆ = b 1 1 + b 2 2 + b 3 3 + b 0 = 0.038Wit + 0.273 Stress 0.074 SocSupp A separate coefficient for each variable These are slopes An intercept (here called b 0 instead of a) + 0.477 Chapter 11 Multiple Regression 27 Chapter 11 Multiple Regression 28 7
Interpretation Note slope for Witness and Stress are positive, but slope for Social Support is negative. Does this make sense? If you had two subjects with identical Stress and SocSupp, a one unit increase in Witness would produce 0.038 unit increase in Internal. Interpretation--cont. The same holds true for other predictors. t test on two slopes are significant SocSupp not significant. Elaborate R 2 : 13.5% of variability in Internal by variability in Witness, Stress, and SocSupp. Chapter 11 Multiple Regression 29 Cont. Chapter 11 Multiple Regression 30 Interpretation--cont. Predictions Intercept usually not meaningful. Prediction when all predictors are 0.0 Assume Witness = 20, Stress = 5, and SocSupp = 35. Yˆ =.038 * Wit +.273 * Stress.074 * SocSupp =.038 ( 20 ) +.273 (5).074 (35 ) + 0.477 Chapter 11 Multiple Regression 31 =.012 Chapter 11 Multiple Regression 32 8
Hypothesis Testing Testing--cont. Test on R 2 given in Analysis of Variance table ANOVA b Sum of Squares df Mean Square F Sig. Regression 454.482 1 454.48 19.59.00 a Residual 440.757 19 23.198 Total 895.238 20 a. Predictors: (Constant), Cigarette Consumption per Adult per Day b. Dependent Variable: CHD Mortality per 10,000 Tests on regression coefficients given along with the coefficients. See next slide Note tests on each coefficient. Chapter 11 Multiple Regression 33 Cont. Chapter 11 Multiple Regression 34 Testing Slopes and Intercept (Constant) Amount violenced witnessed Current stress Social support Coefficients a Unstandardized Coefficients Standardized Coefficients Std. B Error Beta t Sig..477 1.289.37.712.038.018.201 2.1.039.273.106.247 2.6.012 -.074.043 -.166-2.087 a. Dependent Variable: Internalizing symptoms on CBCL Chapter 11 Multiple Regression 35 Moderator Effects in Regression Sometimes you ll find that instead of one overall regression line, you get better prediction if you divide a larger group into smaller subgroups based on a and do regression within each subgroup. Chapter 11 Multiple Regression 36 9
Goldilocks and the 3 Regression Scenarios: Scenario 1 subgroup regression lines Y Goldilocks and the 3 Regression Scenarios: Scenario 2 Group 1 Y Group 1 Group 2 Group 2 overall regression line Intercept Bias Chapter 11 Multiple Regression 37 Slope Bias Chapter 11 Multiple Regression 38 Y Goldilocks and the 3 Regression Scenarios: Scenario 3 Group 1 Same Slopes? Same Intercepts? Moderated Multiple Regression (MMR) To get accurate estimates of slope and intercept differences, you need Group 2 JUST RIGHT The overall regression line works well for both Group 1 and Group 2; predicts similarly regardless of group membership Chapter 11 Multiple Regression 39 for both groups for both groups A construct-valid test measures the same construct in both groups Chapter 11 Multiple Regression 40 10
Summary Multiple regression (MR): more than one predictor, a compensatory model Better predictors in MR: uncorrelated (or low rs) with each other, high correlations with the criterion. Moderated multiple regression: 1 overall regression line (same slope/intercept) or 2 lines(different slopes and/or intercepts)? Review Questions These are NOT the only questions to study How does multiple regression differ from simple regression? Can R 2 decrease as you add predictors? What do we mean by controlling for? How do we calculate our prediction? Is prediction even an important issue in the violence example? Chapter 11 Multiple Regression 41 Chapter 11 Multiple Regression 42 Cont. Review Questions--cont. These are NOT the only questions to study Is it likely that a slope would be significant if the overall R is not? Give an example where multiple regression might help you to understand behavior. Give an example where you might want to conduct moderated multiple regression. Aloha! Chapter 11 Multiple Regression 43 Chapter 11 Multiple Regression 44 11