Math 215, Lab 7: 5/23/2007

Transcription

1 Math 215, Lab 7: 5/23/2007 (1) Parametric versus Nonparamteric Bootstrap. Parametric Bootstrap: (Davison and Hinkley, 1997) The data below are 12 times between failures of airconditioning equipment in a Boeing 720 jet aircraft, for which we wish to estimate the underlying mean or its reciprocal, the failure rate: 3, 5, 7, 18, 43, 85, 91, 98, 100, 130, 120, 230, 487 It makes sense to assume that these data come from an exponential distribution. The problem is we do not know the rate of that distribution hence, we can not identify its pdf. However, we can rely upon the fact that the sample average is an efficient estimator for the exponential mean. Consequently, we may consider an exponential distribution whose rate is the reciprocal of the sample average. Formally, this means: 1) Let y exp(1/µ) f(y) =(1/µ) exp( 1/µ). 2) Let ˆf = fˆµ =(1/ȳ) exp( 1/ȳ). Now, bootstrapping seems obvious here. Obtain a sample Y1,..., Y n from ˆf, thencalculate the statistic of the interest T boot using sampled data, finally repeat this procedure B times to get the distribution for T boot. (a) Perform a parametric bootstrap for the Boeing data to calculate the MSE for the mean, the MSE for the variance, a 95% confidence interval for mean, and a 95% confidence interval for the variance of the exponential distribution of interest. Nonparametric Bootstrap: This is pretty much what we have been doing so far. That is, we tend to assume that the underlying distribution from which the data are generated is unknown. Consequently, we assumed that the data may very well modeled via its empirical distribution which puts equal probabilities of n 1 at each sample value. The latter assumption is equivalent to estimate the unknown cumulative distribution function (CDF) F with its Empirical Cumulative Distribution (ECDF) or ˆF which is defined as the sample proportion: ˆF (y) = #{y j y} n Note that the values of ECDF are fixed and they are : (0, 1, 2,..., n). So in the n n n Nonparametric version, we may sample with replacement from the original dataset B times and proceed as before. (b) Perform a Nonparametric bootstrap for the Boeing data. Carry out calculations similar to part a. 1

2 (2) Consider the following data representing the populations in thousands of n=20 large US cities in 1920 (x) and 1930 (y) (x) 1930 (y) (a) Perform a Nonparametric bootstrap to obtain a distribution for the Pearson s correlation coefficient between the 1920 and 1930 population data. Note in particular that, you need to obtain 20 samples without replacement from the pairs of data. (b) Obtain a 90% confidence interval for the following statistic: Z = 1 2 ln(1+r 1 r ) (3) Consider the bootstrap procedure for obtaining the t-statistic for the slope of a regression line using its associated least-square residuals as explained in page 271. Note that in order to do this, you need to fit e = y ˆβ 0 ˆβ 1 X or a separate regression with e i s being the response values. Subsequently, you need to sample with replacement from e i s to get B repetitions of the slope s t statistic. (a) Use the data of the previous problem to create a 95% bootstrap confidence interval for β 1, the slope of the regression line. 2

3 Multiple Regression Parametric Multiple Regression (Dalgaard, 2002). Dalgaard presents the analysis related to a study concerning lung function in patients with cystic fibrosis. Data are in the ISwR package which can be downloaded from the web. After connecting to the web, all you need to do is to type the following in R: > cyst<-read.table(" > cyst<-as.matrix(cyst) Then, each time you need to use the data sets included in this package in R, you need to click on packages and choose ISwR from the packages menu. Each data set will be available by using the command data (see Dalgaard Appendix A andsection9.1). > cyst age sex height weight bmp fev1 rv frc tlc pemax

4 The description for the data set is also given at page 228 of the text book (Appendix B): The cystfibr data frame has 25 rows and 10 columns. It contains lung function data for for cystic fibrosis patients (7-23 years old). Format: This data frame contains the following columns: age a numeric vector: Age in years. sex a numeric vector code. 0:male, 1:female. height a numeric vector. Height (cm). weight a numeric vector. Weight (kg). bmp a numeric vector. Body mass (% of normal). fev 1 a numeric vector. Forced expiratory volume. rv a numeric vector. Residual volume. frc a numeric vector. Functional residual capacity. tlc a numeric vector. Total lung capacity. pemax a numeric vector. Maximum expiratory pressure. 4

5 Preliminary Exploratory Analysis First of all, by typing: > plot(cystfibr) you can obtain a matrix for all pairwise scatterplots associated with the data set (figure 1). This is an extremely powerful tool in visualizing the multivariate data. secondly, by typing: > attach(cystfibr) the default data set of in R is going to be cystfibr. This is really important because to see the column labeled as height instead of typing > cystfibr[, height] I only need to type: > height. 5

6 age sex height weight bmp fev1 rv frc tlc pemax figure 1. This grid of scatterplots reflects the pairwise relationships between all variables of interest. 6

7 Also, you can get a matrix for all the pairwise correlations by typing: > cor(cystfibr) age sex height weight bmp fev1 age sex height weight bmp fev rv frc tlc pemax rv frc tlc pemax age sex height weight bmp fev rv frc tlc pemax note that this will enable us to browse through all the pairwise linear associations. 7

8 1 Multiple Regression Modeling Here, the response variable is pemax. All the other variables are considered as the independent variables. One might be interested to start with the most complete model. That is, we consider the additive model of: pemax = β 0 + β 1 age + β 2 sex + β 3 height + β 4 weight + β 5 bmp + β 6 fev1+β 7 rv + β 8 frc+ β 9 tlc (1) here is the output for the full model: > summary(lm(pemax~age+sex+height+weight+bmp+fev1+rv+frc+tlc)) Call: lm(formula = pemax ~ age + sex + height + weight + bmp + fev1 + rv + frc + tlc) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) age sex height weight bmp fev rv frc tlc Residual standard error: on 15 degrees of freedom Multiple R-Squared: , Adjusted R-squared: F-statistic: on 9 and 15 DF, p-value: > test<-lm(pemax~age+sex+height+weight+bmp+fev1+rv+frc+tlc) > step(test) Start: AIC= pemax ~ age + sex + height + weight + bmp + fev1 + rv + frc + tlc 8

9 Df Sum of Sq RSS AIC - sex tlc height age frc fev rv <none> weight bmp Step: AIC= pemax ~ age + height + weight + bmp + fev1 + rv + frc + tlc Df Sum of Sq RSS AIC - tlc height age frc rv <none> weight bmp fev Step: AIC= pemax ~ age + height + weight + bmp + fev1 + rv + frc Df Sum of Sq RSS AIC - frc height age rv <none> fev weight bmp Step: AIC= pemax ~ age + height + weight + bmp + fev1 + rv 9

10 Df Sum of Sq RSS AIC - age height rv <none> weight bmp fev Step: AIC= pemax ~ height + weight + bmp + fev1 + rv Df Sum of Sq RSS AIC - height rv <none> weight bmp fev Step: AIC= pemax ~ weight + bmp + fev1 + rv Df Sum of Sq RSS AIC <none> rv bmp fev weight Call: lm(formula = pemax ~ weight + bmp + fev1 + rv) Coefficients: (Intercept) weight bmp fev1 rv

11 The Best Model > summary(lm(formula = pemax ~ weight + bmp + fev1 + rv)) Call: lm(formula = pemax ~ weight + bmp + fev1 + rv) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) weight *** bmp * fev * rv Signif. codes: 0 *** ** 0.01 * Residual standard error: on 20 degrees of freedom Multiple R-Squared: , Adjusted R-squared: F-statistic: on 4 and 20 DF, p-value: > anova(lm(formula = pemax ~ weight + bmp + fev1 + rv)) Analysis of Variance Table Response: pemax Df Sum Sq Mean Sq F value Pr(>F) weight *** bmp fev * rv Residuals Signif. codes: 0 *** ** 0.01 *

12 A Naive Bootstrap Analysis of the Best Model B<-1000 Tboot<-matrix(0,nrow=B,ncol=5) for(i in 1:B) { e.star<-sample(cyst.resid,25,replace=t) y.star<- cyst.predict+e.star Tboot[i,]<-summary(lm(y.star~weight + bmp + fev1 + rv))$coefficients[,1] #Tcov<-summary(lm(y.star~weight + bmp + fev1 + rv))$cov print(i) } par(mfrow=c(2,3)) for(i in 1:5) { hist(tboot[,i]) } > mean(tboot[,1]) [1] > sd(tboot[,1]) [1] > > mean(tboot[,2]) [1] > sd(tboot[,2]) [1] > > mean(tboot[,3]) [1] > sd(tboot[,3]) [1] > > mean(tboot[,4]) [1] > sd(tboot[,4]) [1] > > mean(tboot[,5]) [1] > sd(tboot[,5]) [1]

13 Histogram of Tboot[, i] Histogram of Tboot[, i] Histogram of Tboot[, i] Frequency Frequency Frequency Tboot[, i] Tboot[, i] Tboot[, i] Histogram of Tboot[, i] Histogram of Tboot[, i] Frequency Frequency Tboot[, i] Tboot[, i] figure 2. The Distribution of the Bootstrap Parameter Estimates. 13

14 14