Bayesian and Classical Approaches to Inference and Model Averaging Course Tutors Gernot Doppelhofer NHH Melvyn Weeks University of Cambridge Location Norges Bank Oslo Date 5-8 May 2008
The Course The course provides an introduction to Bayesian inference from the perspectives of a classically trained econometrician. Beginning with Bayes Theorem applied to random parameters, the material examines a number of key issues for classical estimation, and where appropriate considers the Bayesian analog. The material moves from the fundamental dichotomy between fixed and random quantities in classical estimation, and considers the role of the principle distinction between the two approaches - namely random versus fixed parameters. We examine how key notions such as convergence, and the use of simulation as an inference tool differs across the two approaches. The translation of fixed versus random effects panel data models into a Bayesian framework provides a convenient introduction to the use of hierarchical and non-hierarchical priors. The curse of dimensionality which plagues inference in a broad class of latent variable model is used to motivate both the use and development of simulation methods in econometrics. Given fundamental differences in the treatment of missing data between Classical and Bayesian approaches, we consider how the use of Data Augmentation presents a powerful tool to circumvent dimensionality problems in a class of Bayesian models. A number of applications are considered. These are Bayesian model averaging applied to the problem of conducting inference on the nature of financial crises. We also introduce new work on identifying complementarities between policy instruments in estimating model of economic growth. Finally, we apply model averaging to problems of economic forecasting. 2
Course Outline 1. Introduction to Bayesian Inference 2. Bayesian Model Averaging (BMA) and Frequentist Model Averaging 3. Simulation Methods in Classical and Bayesian Modelling 4. Model Averaging: Applications from to financial crises and economic growth 5. Identifying Jointness in the effects of policy instruments 6. Determinants of Financial Crises 7. Forecasting using Model Averaging 3
Bayesian versus Classical Approaches to Inference Agenda Monday 5th May 1 Basics of Bayesian Inference 1. Probability Statements about Unknown Parameters Probability Statements and Interval Estimation 2. Motivation: Multiple Models 3. Bayesian and Classical Objects 4. Binary Uncertainty Bayesian Hypothesis Testing Posterior Odds: A Derivation 5. Bayes Theorem for Events, Random Variables and Parameters 6. Aspects of Bayesian Inference 7. de Finetti s Representation Theorem 8. Pragmatic Bayesians I: Constructing the Posterior II: Specifications of Unobserved Heterogeneity 9. Simultaneous Bayesian and Classical Inference 10. Pointwise Convergence versus Convergence to a Distribution 11. Prior Uncertainty Noninformative and Improper Priors Prior Structures and BMA Natural Conjugate Priors 12. Hierarchical Priors 4
Hierarchical Priors I: Panel Data Hierarchical Priors II: Model Averaging Hierarchical Priors III: Unobserved Heterogeneity in the Returns to Schooling Hierarchical Priors IV: Stochastic Frontier Panel Data Models 5
Tuesday 6th May 2 Model Averaging in the Linear Regression Model 1. Motivation 2. Statistical Framework Decision Theory Unconditional Distribution Bayesian Hypothesis Testing 3. Linear Regression Model Normal Linear Model Likelihood Function Prior Distributions Posterior Analysis Model Space 4. Conclusion 5. Further Readings 6
Wednesday 7th May. 3 Simulation-Based Estimation and Inference 1. Overview 2. Combining Prior and Sample Information 3. Simulation Methods: A Classical Reference Point 4. Simulation Estimation and Discrete Choice Random Coefficient Mixed Logit 5. Simulated Maximum Likelihood estimation The Attraction of the Bayesian Paradigm 6. Bayesian inference in the Binomial Probit Model 7. Data Augmentation with Missing Data Bayesian Analysis of Binary Choice with Data Augmentation Data Augmentation: A General Framework 8. The Integral Transform Theorem 9. Bayesian inference in the Mixed Logit Model 10. Posterior Sampling: Taxonomy 11. Posterior Simulation using MCMC 12. MCMC Methods Gibbs Sampling: Some Specifics 13. Ergodicity 14. The Metropolis Method The Metropolis-Hastings Method 15. Exploring the Model Space The MC3 Algorithm 7
Thursday 8th May. 4 Model Averaging and Applications 1. Introduction 2. Determinants of Economic Growth Model Space Posterior Distribution Jointness Nonlinearities and Thresholds Robustness 3. Epilogue: Robust Policy 8
Principle Texts [1] Albert, J. and S. Chib (1993) Bayesian Analysis of Binary and Polychotomous Response Data. Journal of the American Statistical Association, 88, 669-679. [2] Chib, S.and E. Greenberg (1996) Bayesian Analysis of Multivariate Probit Models, Research Paper, John M. Olin School of Business, Washington University. [3] Koop, G. (2003). Bayesian Econometrics. Wiley. [4] Koop, G. and Poirier, D. J. and Tobias, L. (2007) Bayesian Econometric Methods (Econometric Exercises), Cambridge University Press. [5] Lancaster, T. (2004). An Introduction to Modern Bayesian Econometrics. Blackwell. [6] Train, K. E. (2005) Discrete Choice Methods with Simulation. Cambridge University Press. [7] Mariano, B. M. Weeks, and T. Schuermann (eds) (2000). Simulation Based Inference: Theory and Applications, Cambridge University Press. [8] Mariano, B., Schuermann, T. and M. Weeks. (2002). Simulation-Based Inference in Econometrics: Theory and applications. Cambridge University Press. [9] Van Dijk, H.K., A. Monfort and B.W. Brown (eds) (1995). Econometric Inference Using Simulation Techniques, John Wiley and Sons, Chichester, West Sussex, England. Basics of Bayesian Inference [1] Akaike H. 1973. Information Theory and an Extension of the Maximum Likelihood Principle. In Second International Symposium on Information Theory, Petrov B, Csake F. (eds). Akademiai Kiado: Budapest. [2] Geweke J. 1993. Bayesian Treatment of the Independent Student-t Linear Model. Journal of Applied Econometrics 8: S19-S40. [3] Kass R, Raftery A. 1995. Bayes Factors. Journal of the American Statistical Association 90(430): 773-95. [4] Leamer E.E. 1973. Multicollinearity: A Bayesian Interpretation. Review of Economics and Statistics 55(3): 371-80. 9
Model Averaging in the Linear Regression Model [1] Brock W.A, Durlauf, S.N, West K.D. 2003. Policy Evaluation in Uncertain Economic Environments (with Comments and Discussion). Brookings Papers of Economic Acitivity 1: 235-322. [2] Doppelhofer G. 2008. Model Averaging. Palgrave Dictionary of Economics. 2nd edition. [3] Fernandez C, Ley, E., Steel, M. 2001b. Benchmark Priors for Bayesian Model Averaging. Journal of Econometrics 100(2): 381-427. [4] Hansen B. 2007. Least Squares Model Averaging. Econometrica 75(4): 1175-89. [5] Hjort N., Claeskens G. 2003. Frequentist Model Average Estimators. Journal of the American Statistical Association 98(464): 879-99. [6] Hoeting J., Madigan D, Raftery A., Volinsky C.T. 1999. Bayesian Model Averaging: A Tutorial. Statistical Science 14(4): 382-417. [7] Koop G. 2003. Bayesian Econometrics. Wiley: Chichester. [8] Leamer E.E. 1978. Specification Searches: Ad Hoc Inference with Nonexperimental Data. Wiley: New York. [9] Poirier D.J. 1995. Intermediate Statistics and Econometrics. MIT: Mass. [10] Wasserman L. 2000. Bayesian Model Selection and Model Averaging. Journal of Mathematical Psychology 44(1): 92-107. Simulation-Based Estimation and Inference [1] Albert and Chib (1993) - Probit [2] McCullogh and Rossi (1994) - Multinomial Probit [3] Koop and Poirier (1996) - Nested Logit [4] Mariano, Schuermann and Weeks (2000) - Bayesian Simulation Methods [5] Allenby (1997) - Mixed Logit [6] Chib and Greenberg (1996) - Interrelated Discrete Response [7] Train (2000) - Classical versus Bayesian simulation 10
[8] Koop (2003) - Bayesian Econometrics [9] Lancaster (2005) - An Introduction to Modern Bayesian Econometrics Model Averaging and Applications [1] Brock W.A, Durlauf, S.N, West, K.D. 2003. Policy Evaluation in Uncertain Economic Environments (with Comments and Discussion). Brookings Papers of Economic Acitivity 1: 235-322. [2] Crespo-Cuaresma, J., Doppelhofer G. 2007. Nonlinearities in Cross-Country Growth Regressions: A Bayesian Averaging of Thresholds (BAT) Approach. J. Macroeconomics 29: 541-54. [3] Doppelhofer, G., Weeks, M. (Forthcoming). Jointness of Growth Determinants. Journal of Applied Econometrics. [4] Fernandez C, Ley, E., Steel, M.F.J. 2001a. Model Uncertainty in Cross-Country Regression. J. Applied Econometrics 16: 563-76. [5] Sala-i-Martin, X, Doppelhofer, G., Miller, R.I. 2004. Determinants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach. American Economic Review 94(4): 813-35. Prior Uncertainty [1] Eicher, T.S., C. Papageorgiou, and A.E. Raftery (2007) Determining Growth Determinants: Default Priors and Predictive Performance in Bayesian Model Averaging. Paper, Department of Economics University of Washington. [2] Fernandez C, Ley E, Steel MFJ. 2001b. Benchmark Priors for Bayesian Model Averaging. Journal of Econometrics 100(2): 381-427. [3] Ley, E. and M.F.J. Steel (2008) On the Effect of Prior Assumptions in Bayesian Model Averaging with Applications to Growth Regression MPRA Paper No. 6637. 11