Gianluca Baio. University College London Department of Statistical Science.

Transcription

1 Bayesian hierarchical models and recent computational development using Integrated Nested Laplace Approximation, with applications to pre-implantation genetic screening in IVF Gianluca Baio University College London Department of Statistical Science (Thanks to Sioban Sen Gupta and Anastasia Mania, UCL Centre for PGD) MRC Biostatistics Unit Seminars Cambridge, Tuesday 16 October 2012 Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

2 Outline of presentation 1 Motivation: Modelling the effect of telomere length on chromosomal abnormalities In vitro fertilisation Pre-implantation genetic screening Hierarchically structured data 2 (Bayesian) Hierarchical Models General framework INLA 3 Modelling the effect of telomere length on chromosomal abnormalities Model description Results Discussion Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

3 Outline of presentation 1 Motivation: Modelling the effect of telomere length on chromosomal abnormalities In vitro fertilisation Pre-implantation genetic screening Hierarchically structured data 2 (Bayesian) Hierarchical Models General framework INLA 3 Modelling the effect of telomere length on chromosomal abnormalities Model description Results Discussion Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

4 Outline of presentation 1 Motivation: Modelling the effect of telomere length on chromosomal abnormalities In vitro fertilisation Pre-implantation genetic screening Hierarchically structured data 2 (Bayesian) Hierarchical Models General framework INLA 3 Modelling the effect of telomere length on chromosomal abnormalities Model description Results Discussion Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

5 In vitro fertilisation (IVF) IVF is a process in which egg cells are fertilised by sperm outside the body Eggs are collected from the woman (via minor surgery) and are fertilised by direct injection of the sperm Fertilised eggs (or embryos) are cultured until they reach 6-8 cells (usually 2/3 days after fertilisation). Often, culture continues up to 5/6 days until the embryo reaches the stage of blastocysts The quality of the embryos is visually inspected and the best is transferred in the woman s womb If the embyro implants to the the wall of the uterus, pregnancy starts Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

6 In vitro fertilisation (IVF) IVF is a process in which egg cells are fertilised by sperm outside the body Eggs are collected from the woman (via minor surgery) and are fertilised by direct injection of the sperm Fertilised eggs (or embryos) are cultured until they reach 6-8 cells (usually 2/3 days after fertilisation). Often, culture continues up to 5/6 days until the embryo reaches the stage of blastocysts The quality of the embryos is visually inspected and the best is transferred in the woman s womb If the embyro implants to the the wall of the uterus, pregnancy starts The chance of getting pregnant varies from 4 to 40% Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

7 In vitro fertilisation (IVF) IVF is a process in which egg cells are fertilised by sperm outside the body Eggs are collected from the woman (via minor surgery) and are fertilised by direct injection of the sperm Fertilised eggs (or embryos) are cultured until they reach 6-8 cells (usually 2/3 days after fertilisation). Often, culture continues up to 5/6 days until the embryo reaches the stage of blastocysts The quality of the embryos is visually inspected and the best is transferred in the woman s womb If the embyro implants to the the wall of the uterus, pregnancy starts The chance of getting pregnant varies from 4 to 40% Couples with Advanced maternal age (>40 years old) History of repeated miscarriage Repeated failed IVF cycles (>3 times) are less likely to get pregnant and therefore are usually referred for genetic screening before the embryo transfer Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

8 Pre-implantation genetic screening (PGS) Checks the chromosomes of embryos conceived by IVF for common abnormalities (eg number and size of each chromosome) Diploid state: 23 chromosomes from each parent (normal cells) Anueploid state: abnormal number of chromosomes (eg trisomy 21 causes Down s syndrome) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

9 Pre-implantation genetic screening (PGS) Checks the chromosomes of embryos conceived by IVF for common abnormalities (eg number and size of each chromosome) Diploid state: 23 chromosomes from each parent (normal cells) Anueploid state: abnormal number of chromosomes (eg trisomy 21 causes Down s syndrome) 1 Remove a single cell from the 6-8 cell embryo Usually safe for the embryo Each cell is called a blastomere Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

10 Pre-implantation genetic screening (PGS) Checks the chromosomes of embryos conceived by IVF for common abnormalities (eg number and size of each chromosome) Diploid state: 23 chromosomes from each parent (normal cells) Anueploid state: abnormal number of chromosomes (eg trisomy 21 causes Down s syndrome) 1 Remove a single cell from the 6-8 cell embryo Usually safe for the embryo Each cell is called a blastomere 2 Fluorescent in-situ Hybridization (FISH) Chromosome painting, using fluorescent probes, specific for each chromosome Allows the number and size of each chromosome to be checked Useful for identifying aneuploids and translocations (eg gene fusions) Destroys the tested cells Limited number of chromosomes can be checked simultanously Some abnormalities undetectable Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

11 Pre-implantation genetic screening (PGS) FISH on chromosomes 16 (cyan dye) and chromosome 15 (pink dye) Trisomy 16 Monosomy 15 Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

12 Telomeres There are many possible reasons for chromosomal abnormalities one theory links these to telomeres Telomeres are 6 base pair repeats (TTAGGG) found at the ends of chromosomes to protect them from degradation and subsequent DNA repair Participate in processes of chromosomal repair Prevent non-specific chromosomal recombination (eg fusion with neighboring chromosomes) Allow the complete replication of chromosomal DNA without the inherent loss due to the DNA polymerase machinery Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

13 Telomeres There are many possible reasons for chromosomal abnormalities one theory links these to telomeres Telomeres are 6 base pair repeats (TTAGGG) found at the ends of chromosomes to protect them from degradation and subsequent DNA repair Participate in processes of chromosomal repair Prevent non-specific chromosomal recombination (eg fusion with neighboring chromosomes) Allow the complete replication of chromosomal DNA without the inherent loss due to the DNA polymerase machinery Telomeres get shorter after each cell division, contributing to cellular senescence This limits the regenerative capacity of cells and is correlated with the onset of cancer, ageing and chronic diseases Limited evidence in animals seems to suggest a relationship between the length of the telomere and chromosome mal-segregation, leading to aneuploidy Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

14 Data (collected at the UCL Centre for PGD) Data are available for N = 1130 blastomeres from J = 35 embryos donated by K = 7 couples Cell Embryo Couple PGS Telomere Embryo type Morphology Referral Paternal (i) (j) (k) test length (kb) age PGS test: 1 = abnormal, 0 = normal Embryo type: T1 = Major chaotic, T2 = Minor chaotic, T3 = Major mosaic, T4 = Minor mosaic, T5 = Uniformally abnormal Embryo morphology: M1 = Arrested development, M2 = Blastocyst, M3 = Morula Reason for couples referral: R1 = Repeated miscarriage, R2 = Advanced maternal age, R3 = Unsuccessful IVF Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

15 Data (collected at the UCL Centre for PGD) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

16 Bayesian hierarchical models Hierarchical structures are particularly easy to represent under a Bayesian framework Parameters θ are considered as random variables, whose probability distribution depends on some hyper-parameters ψ In turn, ψ are also considered random variables Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

17 Bayesian hierarchical models Hierarchical structures are particularly easy to represent under a Bayesian framework Parameters θ are considered as random variables, whose probability distribution depends on some hyper-parameters ψ In turn, ψ are also considered random variables Usually, we can assume that the parameters are described by a Gaussian Markov Random Field (GMRF) θ Normal(µ,Σ) θ l θ m θ lm Q lm =: Σ 1 lm = 0 where The notation lm indicates all the other elements of the parameters vector, excluding elements l and m The matrix Σ depends on the hyper-parameters ψ and is generally sparse, given the underlying conditional independence structure Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

18 GMRFs, hierarchical models et al A very general way of specifying the problem is by modelling the mean for the i-th unit by means of an additive linear predictor, defined on a suitable scale (e.g. logistic for binomial data) η i = α+ M β m x mi + L f l (z li ) m=1 l=1 where α is the intercept; β = (β 1,...,β M) quantify the effect of x = (x 1,...,x M) on the response; f = {f 1( ),...,f L( )} is a set of functions defined in terms of some covariates z = (z 1,...,z L) θ = (α,β,f) GMRF(ψ) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

19 GMRFs, hierarchical models et al A very general way of specifying the problem is by modelling the mean for the i-th unit by means of an additive linear predictor, defined on a suitable scale (e.g. logistic for binomial data) η i = α+ M β m x mi + L f l (z li ) m=1 l=1 where α is the intercept; β = (β 1,...,β M) quantify the effect of x = (x 1,...,x M) on the response; f = {f 1( ),...,f L( )} is a set of functions defined in terms of some covariates z = (z 1,...,z L) θ = (α,β,f) GMRF(ψ) NB: Upon varying the form of the functions f l ( ), this formulation can accommodate a wide range of models Standard and hierarchical regression Spatial and spatio-temporal models Time series Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

20 Bayesian hierarchical models Advantages Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

21 Bayesian hierarchical models Advantages Can handle rich structures in a relatively easy way Different distributional forms for the observed data and / or the random effects Correlation between varying intercepts and varying slopes Covariates at different levels Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

22 Bayesian hierarchical models Advantages Can handle rich structures in a relatively easy way Different distributional forms for the observed data and / or the random effects Correlation between varying intercepts and varying slopes Covariates at different levels Better characterisation of the hyper-parameters REML estimation generally produces under-estimation of the variability in the random effects, which leads to interval estimations that are artificially narrow In contrast, the Bayesian procedure accounts directly for the underlying uncertainty in the distribution of the variances Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

23 Bayesian hierarchical models Advantages Can handle rich structures in a relatively easy way Different distributional forms for the observed data and / or the random effects Correlation between varying intercepts and varying slopes Covariates at different levels Better characterisation of the hyper-parameters REML estimation generally produces under-estimation of the variability in the random effects, which leads to interval estimations that are artificially narrow In contrast, the Bayesian procedure accounts directly for the underlying uncertainty in the distribution of the variances Cooler! Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

24 Bayesian hierarchical models Disadvantages The standard way of obtaining the posterior distributions is to perform MCMC computation, which is generally very intensive, especially for hierarchical models Thus alternative methods of estimation are valuable Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

25 Bayesian hierarchical models Disadvantages The standard way of obtaining the posterior distributions is to perform MCMC computation, which is generally very intensive, especially for hierarchical models Thus alternative methods of estimation are valuable Recently, interesting methods have been proposed Hamiltonian Monte Carlo / No U Turn Sampler Integrated Nested Laplace Approximation (INLA) These are generally very effective, thus reducing the computational limitations of standard MCMC Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

26 Integrated Nested Laplace Approximation (INLA) Laplace approximation is a generic method that can be used to approximate a probability distribution g(x) using a Taylor s series expansion up to the quadratic terms It can be shown to be equivalent to using a Normal distribution centered on the mode ˆx and with variance proportional to the second derivative of the function g evaluated at the mode, [g (x)] 1/2 x=ˆx Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

27 Integrated Nested Laplace Approximation (INLA) Laplace approximation is a generic method that can be used to approximate a probability distribution g(x) using a Taylor s series expansion up to the quadratic terms It can be shown to be equivalent to using a Normal distribution centered on the mode ˆx and with variance proportional to the second derivative of the function g evaluated at the mode, [g (x)] 1/2 x=ˆx We can approximate a generic conditional (posterior) distribution as p(z w) = p(x,z w) p(x z,w), where p(x z, w) is the Laplace approximation to the conditional distribution of x given z,w Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

28 Integrated Nested Laplace Approximation (INLA) Objective of Bayesian estimation In a hierarchical models, the required distributions are p(θ j y) = p(θ j,ψ y)dψ = p(ψ y)p(θ j ψ,y)dψ p(ψ k y) = p(ψ y)dψ k Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

29 Integrated Nested Laplace Approximation (INLA) Objective of Bayesian estimation In a hierarchical models, the required distributions are p(θ j y) = p(θ j,ψ y)dψ = p(ψ y)p(θ j ψ,y)dψ p(ψ k y) = p(ψ y)dψ k Thus we need to estimate: (1.) p(ψ y), from which also all the relevant marginals p(ψ k y) can be obtained; Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

30 Integrated Nested Laplace Approximation (INLA) Objective of Bayesian estimation In a hierarchical models, the required distributions are p(θ j y) = p(θ j,ψ y)dψ = p(ψ y)p(θ j ψ,y)dψ p(ψ k y) = p(ψ y)dψ k Thus we need to estimate: (1.) p(ψ y), from which also all the relevant marginals p(ψ k y) can be obtained; (2.) p(θ j ψ,y), which is needed to compute the marginal posterior for the parameters Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

31 Integrated Nested Laplace Approximation (INLA) (1.) can be easily estimated as p(ψ y) = p(θ,ψ y) p(θ ψ,y) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

32 Integrated Nested Laplace Approximation (INLA) (1.) can be easily estimated as p(ψ y) = = p(θ,ψ y) p(θ ψ,y) p(y θ,ψ)p(θ,ψ) p(y) 1 p(θ ψ,y) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

33 Integrated Nested Laplace Approximation (INLA) (1.) can be easily estimated as p(ψ y) = = = p(θ,ψ y) p(θ ψ,y) p(y θ,ψ)p(θ,ψ) 1 p(y) p(θ ψ,y) p(y θ)p(θ ψ)p(ψ) 1 p(y) p(θ ψ,y) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

34 Integrated Nested Laplace Approximation (INLA) (1.) can be easily estimated as p(ψ y) = = = p(θ,ψ y) p(θ ψ,y) p(y θ,ψ)p(θ,ψ) 1 p(y) p(θ ψ,y) p(y θ)p(θ ψ)p(ψ) 1 p(y) p(θ ψ,y) p(ψ)p(θ ψ)p(y θ) p(θ ψ,y) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

35 Integrated Nested Laplace Approximation (INLA) (1.) can be easily estimated as p(ψ y) = = = p(θ,ψ y) p(θ ψ,y) p(y θ,ψ)p(θ,ψ) 1 p(y) p(θ ψ,y) p(y θ)p(θ ψ)p(ψ) 1 p(y) p(θ ψ,y) p(ψ)p(θ ψ)p(y θ) p(θ ψ,y) p(ψ)p(θ ψ)p(y θ) p(θ ψ,y) =: p(ψ y) θ=θ (ψ) where p(θ ψ,y) is the Laplace approximation of p(θ ψ,y) θ (ψ) is its mode Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

36 Integrated Nested Laplace Approximation (INLA) (2.) is slightly more complex, because in general there will be more elements in θ than there are in ψ and thus this computation is more expensive Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

37 Integrated Nested Laplace Approximation (INLA) (2.) is slightly more complex, because in general there will be more elements in θ than there are in ψ and thus this computation is more expensive One easy possibility is to approximate p(θ j ψ,y) directly using a Normal distribution, where the precision matrix is based on the Cholesky decomposition of the precision matrix Q. While this is very fast, the approximation is generally not very good Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

38 Integrated Nested Laplace Approximation (INLA) (2.) is slightly more complex, because in general there will be more elements in θ than there are in ψ and thus this computation is more expensive One easy possibility is to approximate p(θ j ψ,y) directly using a Normal distribution, where the precision matrix is based on the Cholesky decomposition of the precision matrix Q. While this is very fast, the approximation is generally not very good Alternatively, we can write θ = {θ j,θ j }, use the definition of conditional probability and again Laplace approximation to obtain p(θ j ψ,y) = p({θ j,θ j } ψ,y) p(θ j θ j,ψ,y) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

39 Integrated Nested Laplace Approximation (INLA) (2.) is slightly more complex, because in general there will be more elements in θ than there are in ψ and thus this computation is more expensive One easy possibility is to approximate p(θ j ψ,y) directly using a Normal distribution, where the precision matrix is based on the Cholesky decomposition of the precision matrix Q. While this is very fast, the approximation is generally not very good Alternatively, we can write θ = {θ j,θ j }, use the definition of conditional probability and again Laplace approximation to obtain p(θ j ψ,y) = p({θ j,θ j } ψ,y) p(θ j θ j,ψ,y) = p({θ j,θ j },ψ y) p(ψ y) 1 p(θ j θ j,ψ,y) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

40 Integrated Nested Laplace Approximation (INLA) (2.) is slightly more complex, because in general there will be more elements in θ than there are in ψ and thus this computation is more expensive One easy possibility is to approximate p(θ j ψ,y) directly using a Normal distribution, where the precision matrix is based on the Cholesky decomposition of the precision matrix Q. While this is very fast, the approximation is generally not very good Alternatively, we can write θ = {θ j,θ j }, use the definition of conditional probability and again Laplace approximation to obtain p(θ j ψ,y) = p({θ j,θ j } ψ,y) p(θ j θ j,ψ,y) p(θ,ψ y) p(θ j θ j,ψ,y) = p({θ j,θ j },ψ y) p(ψ y) 1 p(θ j θ j,ψ,y) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

41 Integrated Nested Laplace Approximation (INLA) (2.) is slightly more complex, because in general there will be more elements in θ than there are in ψ and thus this computation is more expensive One easy possibility is to approximate p(θ j ψ,y) directly using a Normal distribution, where the precision matrix is based on the Cholesky decomposition of the precision matrix Q. While this is very fast, the approximation is generally not very good Alternatively, we can write θ = {θ j,θ j }, use the definition of conditional probability and again Laplace approximation to obtain p(θ j ψ,y) = p({θ j,θ j } ψ,y) p(θ j θ j,ψ,y) p(θ,ψ y) p(θ j θ j,ψ,y) = p({θ j,θ j },ψ y) p(ψ y) p(ψ)p(θ ψ)p(y θ) p(θ j θ j,ψ,y) 1 p(θ j θ j,ψ,y) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

42 Integrated Nested Laplace Approximation (INLA) (2.) is slightly more complex, because in general there will be more elements in θ than there are in ψ and thus this computation is more expensive One easy possibility is to approximate p(θ j ψ,y) directly using a Normal distribution, where the precision matrix is based on the Cholesky decomposition of the precision matrix Q. While this is very fast, the approximation is generally not very good Alternatively, we can write θ = {θ j,θ j }, use the definition of conditional probability and again Laplace approximation to obtain p(θ j ψ,y) = p({θ j,θ j } ψ,y) p(θ j θ j,ψ,y) p(θ,ψ y) p(θ j θ j,ψ,y) p(ψ)p(θ ψ)p(y θ) p(θ j θ j,ψ,y) = p({θ j,θ j },ψ y) p(ψ y) 1 p(θ j θ j,ψ,y) p(ψ)p(θ ψ)p(y θ) p(θ j θ j,ψ,y) =: p(θ j ψ,y) θ j=θ j (θj,ψ) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

43 Integrated Nested Laplace Approximation (INLA) Because (θ j θ j,ψ,y) are reasonably Normal, the approximation works generally well However, this strategy can be computationally expensive Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

44 Integrated Nested Laplace Approximation (INLA) Because (θ j θ j,ψ,y) are reasonably Normal, the approximation works generally well However, this strategy can be computationally expensive Thus, the most efficient algorithm is the Simplified Laplace Approximation Based on a Taylor s series expansion of the the Laplace Approximation Can be corrected including mixing terms (e.g. spline) to increase the fit to the required distribution Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

45 Integrated Nested Laplace Approximation (INLA) Because (θ j θ j,ψ,y) are reasonably Normal, the approximation works generally well However, this strategy can be computationally expensive Thus, the most efficient algorithm is the Simplified Laplace Approximation Based on a Taylor s series expansion of the the Laplace Approximation Can be corrected including mixing terms (e.g. spline) to increase the fit to the required distribution This is the algorithm implemented by default by R-INLA, but this choice can be modified Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

46 Integrated Nested Laplace Approximation (INLA) Operationally, the INLA algorithm proceeds with the following steps: Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

47 Integrated Nested Laplace Approximation (INLA) Operationally, the INLA algorithm proceeds with the following steps: i. Explore the marginal joint posterior for the hyper-parameters p(ψ y) Locate the mode of the distribution; Perform a grid search and produce a set of relevant points {ψ k }, together with a corresponding set of weights { k } Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

48 Integrated Nested Laplace Approximation (INLA) Operationally, the INLA algorithm proceeds with the following steps: i. Explore the marginal joint posterior for the hyper-parameters p(ψ y) Locate the mode of the distribution; Perform a grid search and produce a set of relevant points {ψ k }, together with a corresponding set of weights { k } ii. For each element ψ k, Obtain the marginal posterior p(ψ k y), using interpolation and possibly correcting for (probable) skewness by using log-splines; Evaluate the conditional posteriors p(θ j ψ k,y) on a grid of selected values for θ j; Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

49 Integrated Nested Laplace Approximation (INLA) Operationally, the INLA algorithm proceeds with the following steps: i. Explore the marginal joint posterior for the hyper-parameters p(ψ y) Locate the mode of the distribution; Perform a grid search and produce a set of relevant points {ψ k }, together with a corresponding set of weights { k } ii. For each element ψ k, Obtain the marginal posterior p(ψ k y), using interpolation and possibly correcting for (probable) skewness by using log-splines; Evaluate the conditional posteriors p(θ j ψ k,y) on a grid of selected values for θ j; iii. Marginalise ψ k to obtain the marginal posteriors p(θ j y) using numerical integration K p(θ j y) p(θ j ψ k,y) p(ψ k y) k k=1 Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

50 Integrated Nested Laplace Approximation (INLA) So, it s all in the name... Integrated Nested Laplace Approximation Because Laplace approximation is the basis to estimate the unknown distributions Because the Laplace approximations are nested within one another Since (2.) is needed to estimate (1.) NB: Consequently the estimation of (1.) might not be good enough, but it can be refined Because the required marginal posterior distributions are obtained by (numerical) integration Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

51 Integrated Nested Laplace Approximation (INLA) So, it s all in the name... Integrated Nested Laplace Approximation Because Laplace approximation is the basis to estimate the unknown distributions Because the Laplace approximations are nested within one another Since (2.) is needed to estimate (1.) NB: Consequently the estimation of (1.) might not be good enough, but it can be refined Because the required marginal posterior distributions are obtained by (numerical) integration Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

52 Integrated Nested Laplace Approximation (INLA) So, it s all in the name... Integrated Nested Laplace Approximation Because Laplace approximation is the basis to estimate the unknown distributions Because the Laplace approximations are nested within one another Since (2.) is needed to estimate (1.) NB: Consequently the estimation of (1.) might not be good enough, but it can be refined Because the required marginal posterior distributions are obtained by (numerical) integration Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

53 Integrated Nested Laplace Approximation (INLA) So, it s all in the name... Integrated Nested Laplace Approximation Because Laplace approximation is the basis to estimate the unknown distributions Because the Laplace approximations are nested within one another Since (2.) is needed to estimate (1.) NB: Consequently the estimation of (1.) might not be good enough, but it can be refined Because the required marginal posterior distributions are obtained by (numerical) integration Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

54 Hierarchical model for the telomere data Objectives of the model: Investigate the association between chromosomal abnormalities y = (0, 1) and telomere length L Account for the effect of Possible confounders (embryo and couple characteristics) The hierarchical structure (blastomeres clustered in embryos, clustered by couples) NB To ease interpretation, we consider L i = L i L and P k = P k P Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

55 Hierarchical model for the telomere data Objectives of the model: Investigate the association between chromosomal abnormalities y = (0, 1) and telomere length L Account for the effect of Possible confounders (embryo and couple characteristics) The hierarchical structure (blastomeres clustered in embryos, clustered by couples) NB To ease interpretation, we consider L i = L i L and P k = P k P Thus we model y i Bernoulli(π i ) [level 1 model] logit(π i ) = α 0 +α 1 L i +u j[i] [NB: u j[i] = f 1 (embryo i )] Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

56 Hierarchical model for the telomere data Objectives of the model: Investigate the association between chromosomal abnormalities y = (0, 1) and telomere length L Account for the effect of Possible confounders (embryo and couple characteristics) The hierarchical structure (blastomeres clustered in embryos, clustered by couples) NB To ease interpretation, we consider L i = L i L and P k = P k P Thus we model y i Bernoulli(π i ) [level 1 model] logit(π i ) = α 0 +α 1 L i +u j[i] [NB: u j[i] = f 1 (embryo i )] u j Normal(µ j,τ u ) [level 2 model] 4 µ j = β l T lj +β 5 M j +v k[j] [NB: v k[j] = f 2 (couple j )] l=1 Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

57 Hierarchical model for the telomere data Objectives of the model: Investigate the association between chromosomal abnormalities y = (0, 1) and telomere length L Account for the effect of Possible confounders (embryo and couple characteristics) The hierarchical structure (blastomeres clustered in embryos, clustered by couples) NB To ease interpretation, we consider L i = L i L and P k = P k P Thus we model y i Bernoulli(π i ) [level 1 model] logit(π i ) = α 0 +α 1 L i +u j[i] [NB: u j[i] = f 1 (embryo i )] u j Normal(µ j,τ u ) [level 2 model] 4 µ j = β l T lj +β 5 M j +v k[j] [NB: v k[j] = f 2 (couple j )] l=1 v k Normal(δ k,τ v ) δ k = γ 1 R 1k +γ 2 R 2k +γ 3 P k [level 3 model] Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

58 Hierarchical model for the telomere data Model characteristics: The model parameters are θ = (α,β,γ,u,v) The hyper-parameters are the precisions ψ = (τ u,τ v ) Prior specification: Several alternative specifications have been tested, among them Default R-INLA (minimally) informative priors Gamma on the precisions of the structured effects Vague uniform prior on the standard deviations of the structured effects Informative Half-Cauchy on the standard deviations of the structured effects (with MCMC) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

59 Hierarchical model for the telomere data Model characteristics: The model parameters are θ = (α,β,γ,u,v) The hyper-parameters are the precisions ψ = (τ u,τ v ) Prior specification: Several alternative specifications have been tested, among them Default R-INLA (minimally) informative priors Gamma on the precisions of the structured effects Vague uniform prior on the standard deviations of the structured effects Informative Half-Cauchy on the standard deviations of the structured effects (with MCMC) The results are mainly insensitive to different priors, but for some parameters it is necessary to use informative priors to reduce the range of the resulting coefficients (separation) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

60 R-INLA NB: Info, case studies, papers and help on # Installs and loads the library INLA source(" library(inla) # Defines the formula expressing the model formula <- y ~ Lstar+T1+T2+T3+T4+M+R1+R2+Pstar+ f(embryo,model="iid")+f(couple,model="iid") # Calls the inla function to run the model m <- inla(formula,data=data,family="binomial",ntrials=rep(1,n), control.compute=list(dic=true),quantiles=c(.025,.975)) # Summarises the results summary(m) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

61 R-INLA Results Time used: Pre-processing Running inla Post-processing Total Fixed effects: mean sd 0.025quant 0.975quant kld (Intercept) Lstar M T T T T G G Pstar Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

62 R-INLA Results Time used: Pre-processing Running inla Post-processing Total Fixed effects: mean sd 0.025quant 0.975quant kld (Intercept) Lstar M T T T T G G Pstar NB Time to run the equivalent MCMC model on the same computer, with iterations, burn-in and thinning of 180: secs Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

63 R-INLA Results Regression coefficients (unstructured predictors) γ 3 γ 2 γ 1 β 4 Paternal age Advanced maternal age Repeated miscarriage Unsuccessful IVF Uniformally abnormal Minor chaotic β 3 Minor mosaic β 2 Major mosaic β 1 Major chaotic β 5 α 1 α 0 Blastocyst Telomere length (Intercept) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

64 R-INLA Results Pr(chromosomal abnormalities) Major chaotic Major mosaic Minor chaotic Minor mosaic Uniformally abnormal Telomere length (difference from the mean) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

65 R-INLA Results Random effects: Name Model Max KLD embryo IID model couple IID model Model hyperparameters: mean sd 0.025quant 0.975quant Precision for embryo 1.161e e e e+00 Precision for couple 1.881e e e e+04 Expected number of effective parameters(std dev): 22.20(1.331) Number of equivalent replicates : Deviance Information Criterion: Effective number of parameters: Marginal Likelihood: Warning: Interpret the marginal likelihood with care if the prior model is Posterior marginals for linear predictor and fitted values computed Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

66 R-INLA Results Using some built-in INLA functions model$marginals.hyperpar inla.expectation inla.rmarginal it is possible to compute the structured variability, for example on the standard deviation scale, based on nsamples (default=1000) MC simulations s <- inla.contrib.sd(m) s$hyper mean sd 2.5% 97.5% sd for embryo sd for couple Clustering by embryo seems relevant, while clustering by couples is less evident Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

67 R-INLA Results Posterior distributions for the standard deviations of the structured effects Frequency Frequency Standard deviation at embryo level Standard deviation at couple level Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

68 Individual (couple-embryo) analysis Embryos for couple no. 1 Embryos for couple no. 2 Embryos for couple no. 3 Pr(Chromosomal abnormality) Pr(Chromosomal abnormality) Pr(Chromosomal abnormality) Telomere length (difference from the mean) Telomere length (difference from the mean) Telomere length (difference from the mean) Embryos for couple no. 5 Embryos for couple no. 6 Embryos for couple no. 7 Pr(Chromosomal abnormality) Pr(Chromosomal abnormality) Pr(Chromosomal abnormality) Telomere length (difference from the mean) Telomere length (difference from the mean) Telomere length (difference from the mean) Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

69 R-INLA Results Chromosomally abnormal cells are associated with shorter telomeres than normal cells Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

70 R-INLA Results Chromosomally abnormal cells are associated with shorter telomeres than normal cells Minor abnormalities are less likely to produce a positive PGS test when compared to uniformal abnormalities Moreover, for embryos that are relatively good (minor abnormalities), the length of telomere does not have a strong effect on the probability of testing positive to PGS Paternal age has a small and non-significant effect Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

71 R-INLA Results Chromosomally abnormal cells are associated with shorter telomeres than normal cells Minor abnormalities are less likely to produce a positive PGS test when compared to uniformal abnormalities Moreover, for embryos that are relatively good (minor abnormalities), the length of telomere does not have a strong effect on the probability of testing positive to PGS Paternal age has a small and non-significant effect Shorter telomeres could possibly affect embryonic ploidy, quality and survival, which are all vital for pre-implantation embryo development Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

72 Conclusions Integrated Nested Laplace Approximation is a very effective tool to estimate Bayesian hierarchical models Estimation time can be much lower than for standard MCMC Precision of estimation is usually higher than for standard MCMC Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

73 Conclusions Integrated Nested Laplace Approximation is a very effective tool to estimate Bayesian hierarchical models Estimation time can be much lower than for standard MCMC Precision of estimation is usually higher than for standard MCMC But: MCMC still provides a more more flexible approach Virtually any model can be fit using JAGS/BUGS The range of priors available is wider in an MCMC setting than in INLA Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

74 Conclusions Integrated Nested Laplace Approximation is a very effective tool to estimate Bayesian hierarchical models Estimation time can be much lower than for standard MCMC Precision of estimation is usually higher than for standard MCMC But: MCMC still provides a more more flexible approach Virtually any model can be fit using JAGS/BUGS The range of priors available is wider in an MCMC setting than in INLA Nevertheless, several models can be fit in INLA Generalised linear (mixed) models Log-Gaussian Cox processes Survival analysis Spline smoothing Spatio-temporal models Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32

75 Thank you! Gianluca Baio ( UCL) PGS & INLA MRC Cambridge, 16 October / 32