Bayesian Inference Bayes Laplace

Transcription

1 Bayesian Inference Bayes Laplace

2 Course objective The aim of this course is to introduce the modern approach to Bayesian statistics, emphasizing the computational aspects and the differences between the classical and Bayesian approaches. The course includes Bayesian solutions to real problems.

3 Recommended reading Lee, P.M. (2004). Bayesian Statistics: An Introduction, (3 rd ed.), Hodder Arnold. A basic introduction to Bayesian statistics. Bernardo, J.M. and Smith, A.F.M. (1994). Bayesian Theory, Wiley. The new bible of Bayesian statistical theory. Gelman, A., Carlin, J.B., Stern, H. and Rubin, D.B. (2003). Bayesian Data Analysis (2 nd ed.), Chapman and Hall. An applications oriented text. Robert, C.P. and Casella, G. (2004). (2 nd ed.), Springer Verlag. All about Monte Carlo and MCMC methods. Monte Carlo Statistical Methods

4 Course evaluation This will be based on (an exam), some applied coursework and a final applied Bayesian project from an area you are interested in. Some recent course projects have been: Bayesian learning in neural networks Bayesian arbitrage threshold analysis A Glance at Game Theory Bayesian inference for Markovian queues Estimation of objective market potential... You may choose (almost) any Bayesian theme you wish.

5 Course outline 1. Introduction and non-bayesian inference Probability and its interpretations. Subjective probability. Statistical inference. Classical inference. Ideas y criticisms. Other approaches: fiducial inference, likelihood based approaches. The likelihood principle.

6 2. Introduction to Bayesian inference: coin tossing problems Basic elements of Bayesian inference: Bayes theorem and its interpretation Prior and posterior distributions. Likelihood principle. Coin tossing problems: Results with a uniform prior. Prediction. Results with a beta (conjugate) prior. What happens when a non-conjugate prior is used.

7 3. Conjugate families of distributions Conjugate families. Sufficient statistics and exponential families of distributions. Mixtures of conjugate distributions. Applications. Introduction to Monte Carlo sampling.

8 4. Gaussian models Inference for the normal distribution. The use of improper prior distributions. Introduction to Gibbs sampling. Two sample problems. The Behrens Fisher problem. Applications.

9 5. Choosing a prior distribution Subjective prior distributions: Methods for soliciting a subjective prior, Problems with subjective information. Non informative prior distributions: Insufficient reason and uniform priors, Jeffreys prior distributions, Maximum entropy, Reference priors, Problems with the use of improper and non informative prior distributions.

10 6. Implementation of Bayesian inference Numerical integration. Monte Carlo approaches: importance sampling and rejection sampling. MCMC algorithms: Markov chains, The Metropolis Hastings algorithm, Gibbs sampling, Other algorithms, Implementation of Gibbs sampling via WinBugs. Applications.

11 7. Estimation, hypothesis testing and model choice Estimation as a decision problem. Credible intervals and the differences betwen Bayesian and classical intervals. Hypothesis testing: Simple and composite tests. Lindley s paradox. Bayes factors: Approximations, Relation to classical criteria, Problems with improper priors, Generalizations: Intrinsic and fractional Bayes factors. Application.

12 8. Large samples A Bayesian central limit theorem. Applications of the theorem. The Laplace approximation. Applications.

13 9. Regression and linear models Linear models. The two stage linear model. Introduction to hierarchical models and the 3 stage linear model. Generalized linear models. Applications.

14 10. Exchangeability and hierarchical models Exchangeability. De Finetti s theorems. Hierarchical models. Empirical Bayes approaches. Gibbs sampling for hierarchical models. Applications.

15 11. Time series and dynamic linear models Dynamic linear models: The closed, constant DLM, Relations with classical approaches, Modelling trend and seasonality. Other approaches to time series.

16 12. Other topics Selected from: Robustness and sensitivity analysis. Bayesian nonparametric methods. Graphical models, belief nets and Bayes linear models. Decision analysis.