Regularized Joint Estimation of Related VAR Models via Group Lasso

Transcription

1 Research Article Regularized Joint Estimation of Related VAR Models via Grou Lasso Sriniov A *, and Michailidis G Deartment of Mathematics, University of Houston, Houston, Texas, USA Deartment of Statistics, University of Florida, Gainesville, Florida, USA Journal of Biostatistics and Biometric Alications Volume 3 ssue SSN: Oen Access * Corresonding author: Sriniov A, Deartment of Mathematics, University of Houston, 790 Cambridge St. At., Houston, Texas, USA, Tel: , usdandres@gmail.com Citation: Sriniov A, Michailidis G (08) Regularized Joint Estimation of Related VAR Models via Grou Lasso. J Biostat Biometric A 3(): 04 Received Date: November 4, 07 Acceted Date: February 0, 08 Published Date: February 3, 08 Abstract n a number of alications, one has access to high-dimensional time series data on several related subects. Natural examle comes from medical exeriments: brain fmr time series data for various grous of atients, be it controls or individuals with a secific mental disorder. n this wor, we discuss the roblem of their regularized oint estimation, introduced via Vector Autoregressive modeling (VAR) and leveraging a grou lasso enalty on to of regular lasso, so as to increase statistical efficiency of estimates by borrowing strength across the subects. We develo a modeling framewor that allows for both grou level and subect-secific effects for related subects, using grou lasso to estimate the former. Besides a simulation study, we also use our aroach to tacle some of the nown issues of effective connectivity estimation for resting-state fmr data. n articular, a grou-level descritive analysis is conducted for brain inter-regional temoral effects of ADHD and control atients from ADHD-00 Global Cometition, and the findings are comared to those in neuroscience literature. Keywords: Attention deficit hyeractivity disorder; Grou lasso; Regularized estimation; Resting-state fmr; stability selection; Vector autoregression ntroduction With recent advances in technology and growing amounts of available data (clic-generated web browsing data, social networs, image and video data), interest in modeling and analysis of high-dimensional time series data is at its ea. Alication areas include web recommender systems for log data [], brain fmr data [], gene regulatory networ inference [3], macroeconomic time series forecasting and structural analysis [4], to name a few. Their common characteristic is the large number of variable relationshis being analyzed, relative to the time oints available, thus leading to a high-dimensional roblem. n many cases, the temoral dynamics of the data under consideration are well catured by autoregressive models, and hence the use of vector autoregressive models (VAR) enables the modeling of temoral deendencies between the variables. However, in the resence of a large number of arameters to estimate and only few time oints, one needs to incororate aroriate sarsity assumtions into the VAR modeling framewor. To enforce sarsity, the most classic aroach over the years has imlemented lasso regularizing enalty [5], with [6] studying theoretical roerties for regularized estimation of VAR models in articular. n many alications, on to of the tyical high-dimensional setting, one also has to erform estimation of time series across a moderate to large number of related subects. As a motivating examle, the area of medical research brings about a lot of exerimental settings with multile subects being monitored over time, e.g., a collection of fmr time series data for a grou of atients. Looing at atients with a articular disease (Alzheimer s, for examle), we exect their brain connectivity (both functional and effective) to have a certain common structural attern. But at the same time, due to natural variability and subectsecific features, we usually witness certain individual atterns of temoral relationshis for each atient as well. Classic aroach to this roblem of fmr time series analysis is erforming estimation for each atient searately, and subsequently accumulating the estimates for further grou-level analysis. This methodology has been alied to study brain activity for Alzheimer s disease [7], Autism [8], Parinson s disease [9], among others. While roviding tools for grou-level analysis, this aroach does not incororate the rior nowledge of similarity among atients within the same exerimental grou, attributable to their shared mental condition, into the estimation rocedure. The main novelty of this wor is the develoment of oint modeling framewor that would enforce the similarity assumtion into the estimation rocedure, while also leaving room for subect-secific effects estimation. t will allow us to increase effective samle size for common structure estimation by borrowing strength across the related time series. Additionally, the framewor ermits detection of most considerable individual effects of each subect (if resent). The roblem of oint estimation has received attention Annex Publishers Volume 3 ssue

2 Journal of Biostatistics and Biometric Alications in the literature recently, rimarily focusing on the estimation of multile grahical models. Proosed aroaches leveraged various enalties that encouraged both sarsity and oint estimation of the arameters across the models, see the hierarchical enalty used in [0] or fused lasso enalty in []. n this wor we will imlement grou lasso enalty due to its ability to clearly identify a common structure across multile subects, while letting the magnitudes of effects vary. After having detected the common structure, classic sarse lasso rocedure will be alied to obtain subect-secific effect estimates. While the introduced oint estimation rocedure can be used for numerous time series settings (econometric data for cities with shared manufacturing and economic features, gene exression data for matched subects, sales data for similar stores), the rimary intended alication is resting-state fmr time series data for studying the sontaneous brain temoral dynamics of various mental diseases. All the aers on grou-level inference for mental disorders mentioned in the revious aragrah dealt with estimating the functional brain connectivity (inferring the correlations between brain region signals) rather than attemting to infer effective connectivity (the temoral influence across the brain regions). The restingstate data reresents monotone withinbrain fluctuations and classical VAR serves well for caturing the dynamics of such data. Also, we assume each atient s VAR model to be a erturbation of some common underlying VAR model for this exerimental grou (be it subects with disease or healthy controls), the structure of which will be estimated by our oint rocedure via grou lasso. While there has been lenty of deserved critique against using VAR modeling for causal inference in brain neuroimaging studies [,3], we attemt to alleviate some of those issues in our ADHD study. e.g., one of the six roblems for causal inference from fmr discussed in [3] concerned varying signal strength across the brain regions for multile study articiants, even though they all share a common abstract rocessing structure. n our framewor, this issue is addressed directly via grou lasso, which enforces this common structure while leaving room for variability of effect magnitudes. t is also worth mentioning that oint estimation aroach via regularizing enalties had been used before for brain fmr time series data [4,5], but only for functional connectivity estimation, while we aly it for inferring effective connectivity. The remainder of the aer is organized as follows: materials and methods section describes the oint modeling framewor and introduces the two-stage estimation rocedure, results and discussion section demonstrates the erformance of the oint estimation rocedure for both simulated data and the restingstate fmr case study, while the conclusion section contains concluding remars and otential outline of future wor. Materials and Methods Below is the detailed outline of the methodology used to carry out the oint estimation of VAR models sharing common structure. VAR Model Formulation To introduce the oint modeling framewor, consider -variable stationary time series X t =(Xt,...,X t )', t =,...,T for =,...,K related subects. Then the VAR model with lag order D, or VAR(D), is given by where A d is x transition matrix that catures temoral effects of order d between the variables for subect, d=,..., D, =,..., K. Simlifying assumtion of diagonal error covariance matrix σ will allow us to brea roblem () into arallel roblems of lower dimensions. n this wor, we focus on the case of VAR model with lag order one (D = ), so as to emhasize studying the roerties of the oint estimation rocedure rather than the asects of lag order selection. The oint estimation aroach starts with the assumtion of common and individual comonent for each VAR model: A d = Ad,C +Ad,, d =,...,D, =,...,K. Afterwards, a two-stage estimation algorithm is roosed,consisting of grou lasso otimization rocedure to ointly estimate the common d,c comonents {A } of K subects during the first stage, followed by sarse lasso otimization rocedure to estimate the individual d, comonents {A } on the second stage. Grou lasso enalty grous the resective elements of the transition matrices across all K subects and either retains or excludes the whole grou from the model, which guarantees shared structure of resulting common comonent estimates. Meanwhile, the residuals from the common comonent signal are used as data to estimate the individual structures, resresenting subect-secific effects, via classic lasso rocedure. Standard Regression Formulation for VAR Models ( ) X = AX + + A X + ε, ε ~ N 0, σ, t= D,..., T, =,..., K, () t t D t D t t Recalling the system of VAR model equations from (), we will roceed to introduce an equivalent system of standard regression equations. First, we dro from the notation, =,...,K, and show the sequence of required algebraic transformations for a single VAR(D) model: ( ) t t D t X AX A X D ε t, ε t ~ N 0, σ, t= D,..., T = () Our indeendence assumtion for errors fet {ε t, =,...,}, imlied by diagonal error covariance structure cov(ε t ) = σ, t = D,...,T, allows us to reresent temoral dynamics for each of the variables in the form of the following system of equations: ( ) ε ε ( σ ) X = A[,] X + + A [,] X +, ~ N 0,, t = D,..., T, =,...,, t t D t D t t t l l= Annex Publishers Volume 3 ssue (3)

3 3 Journal of Biostatistics and Biometric Alications where A d [,l] is order-d temoral effect of l th variable on th, l, = ;...,. f we let A d [,.] = (A d [,],...,A d [, ]) T ; d =,...,D, then all T - equations from (3) can be reresented in a comact matrix form for each variable resectively: T T T T D T-D X X... X... X... X T A[,.] ε... = D D D- 0 X 0 D D... X... X X... X A [,. ] ε D B A[,. ] ε X B X = B A [,.] + ε, ε ~ N 0, σ T, =,... ( T D+ ) ( T D+ ) D D ( T D+ ) ( ) (4) Now, reinstating, =,...,K in the notation and using the standard regression reresentation (4) for all K VAR models under consideration, we get: where A [,.] corresonds to temoral effects (of all D orders) that each of variables has on th variable for th subect, =,...,K. Common and ndividual Comonent [ ] X, = BKA,. + ε,, ε, ~ N(0, σ T), =, X K, = BKA K[,. ] + εk,, εk, ~ N(0, σkt), =,... The ey assumtion we mae is that of shared structure among the related subects. n our model it is manifested in similar sarsity attern across K transition matrices within the same grou. Additionally, one has to account for the inevitable heterogeneity introduced by natural variability among subects under consideration, leading to certain subect-secific effects. Both of these asects are catured by breaing down each subect s transition matrices into two arts: (5) A = A + A, d=,..., D, =,... K, d dc, d, (6) where A d,c is the common comonent of order-d temoral effects for subect, A d, - individual comonent. Alying this reresentation to equations in (5) we get: Assuming each of K related VAR model to be a erturbation of a common underlying VAR model, we enforce the common suort constraint on {A C [,.],...,A KC [,.] }. Moreover, sarsity assumtion is imosed on the individual comonents {A [,.],...,A K [,.]} d,c d, to account for most imortant subect-secific effects. Lastly, orthogonality constraint A A is introduced, imlying that d,c d, suort intersection for A and A is an emty set, d =,...,D, = ;...,K. n combination with shared suort of common comonents, orthogonality guarantees identifiability of the individual comonent. The full set of constraints for system (7) is described below: Estimation Procedure: Two-Stage Aroach To enforce the set (8) of aforementioned constraints we will imlement an estimation algorithm with each iteration consisting of two stages. During first stage, the common comonent is estimated via grou lasso, while second stage uses the residuals of that common comonent estimate as data for sarse estimation of individual comonents. n order to guarantee orthogonality of resulting common and individual comonent estimates, we automatically eliminate the effects selected during first stage from consideration for second stage rocedure. Consistent maximum lielihood estimators { ˆ σ, =,..., K} will be calculated as in [6] for the system of equations in (), and subsequently lugged in for { σ, =,..., K} in (7). Let us set x C = (A C [,.],...,A KC [,.]) T, ( ) x = (A [,.],...,A K [,.]) T, X = X,,..., X K,, =,...,,, Dˆ ˆ σ = Diag ˆ σ ˆ ˆ ˆ ˆ ˆ,..., σ, σ,..., σ,..., σk,... σ, and ˆB K( T D+ ) K( D ) K, - bloc-diagonal matrix,with th bloc equal to C ( [ ] [ ]) X, = BK A,. + A,. + ε,, ε, ~ N(0, σ T), =, C X K, = BK ( A K[,. ] + AK[,. ]) + εk,, εk, ~ N(0, σkt), =,... [ ] [ ] K [ ] [ ] [ ] [ ] C C C A,. ~ A,. ~ ~A,., =,..., A,. sarse, =,..., K, =,..., C A,. A,., =,..., K, =,..., T D+ T D+ T D+ ( T D+ ) ( D ) B from Section 5., =,...,K. Annex Publishers Volume 3 ssue (7) (8)

4 Journal of Biostatistics and Biometric Alications 4 Two-Stage Estimation Algorithm (for arbitrary, =,..., ). nitialize ˆx with a ozero-vector, use maximum lielihood estimates ˆ σ for σ, =,..., K.. First stage: To enforce the similarity constraint from (8) for common comonents within vector x C, use the following convex grou lasso otimization criterion ˆ C G C C D ˆ ( X ˆ ) ( ) ( ) C x σ λ i Ki i= min Bx Bx x,..., x, 3. First stage (continued): After calculating the solution ath for (9) [7], ic the final common comonent estimate via Bayesian nformation Criterion (BC) ˆ C BC( λ ) nlog ˆ ( Bxˆ ˆ = D X Bx ( λ) ) + log(n)df σ λ (0), (9) where n = K(T -D+), ˆx C ( ) ˆx C ( λ ), calculated as in [8]. λ - estimate corresonding to value λ of the solution ath, dfλ - degrees of freedom for estimate 4. Second stage: Let ˆx C denote the estimate of x C from the first stage. To enforce the ˆ ˆ x C x constraint, let B - matrix containing columns of B corresonding to zero elements of ˆx C. With residuals of ˆx C as resonse, use the following convex lasso roblem to estimate x J ˆ C C SPARS D ˆ ( X ˆ C ) + λ x σ min Bx Bx x, () SPARS 5. Second stage (continued): Similarly to Ste 3, use BC to ic the final estimate ˆx ( ˆ λ ).After selecting the tuning arameter SPARS value ˆ λ, we comlement the resulting vector ˆx ( ˆSPARS λ ) with zeros to a full individual comonent estimate ˆx. C 6. f it is the second iteration (or higher): denote xˆ ˆ ˆ = x + x as the full estimate for current iteration, and ˆx r - full estimate from r revious iteration; sto the algorithm if xˆ ˆ x < 0. Otherwise, go bac to Ste (First stage), using ˆx calculated during Ste 5. Results and Discussion Simulation Study As mentioned in the introduction, we will emhasize studying VAR models of order D= in order to focus on the roerties of oint estimation rocedure rather than on model order selection: ( ) X = AX + ε, ε ~ N 0, σ, t=,..., T, =,..., K, () t t t t Evaluation of the Estimation Aroach: We will evaluate our two-stage estimation aroach by generating a grou of matrices {A =A C K +A,=,...,K} such that C... C C A = = AK A and A is an arbitrary sarse matrix, =,...,K. Denoting estimated matrix as Aˆ = ( aˆ i, ) and the true matrix as A = ( aˆ ) i,, we use the following metrics to evaluate erformance of our estimation rocedure: False Positive (FP) and True Negative (TN) rates: False Negative (FN) and True Positive (TP) rates: ( ˆ i, = ai, ) ( ai, = 0) < i FP = TN FP < i ( ˆ i, ai, = ) ( ai, 0) a < i FN = TP FN < i 0, 0, =-, 0, 0, =-, Matthews Correlation Coefficient (MC, geometric mean of FP and FN): a MC = TP TN FP FN ( TP + FP)( TP + FN )( TN + FP)( TN + FN ). Annex Publishers Volume ssue

5 5 Journal of Biostatistics and Biometric Alications We calculate all three of these measures for common comonent estimates  C and individual comonent estimates Â, =,....,K. Low values of FP and FN (near 0) and high values of MC (near ) would indicate good erformance. Additionally, an average number of iterations needed for convergence is rovided for the two-stage estimation algorithm described above. Simulation Results and Discussion: Matrices (A = A C +A ; =,...,K) are generated with sectral radius of 0.4, with non-zero effects having magnitude of at least 0.. This would guarantee stationarity of generated time series, searate noise from the signal, and liely relicate the features of fmr data from ADHD study (no estimated effects had magnitude larger than 0.4). Data is generated with indeendent N(0,) errors and signal-to-noise ratio of one (as in ADHD study). Multile settings are considered by varying numbers of variables er subect, T of observed time oints, and K of subects in a grou. All the diagonals are generated to be non-zero, while the edge density of elements in the common comonent is set to 5% (of off-diagonal elements) for = 0 and % for = 30, 40. The edge density for individual comonent is set to -3% (of total number of matrix elements), imlying a moderate level of heterogeneity. An examle of generated transition matrices for the case of = 0 can be found in the sulement. Cases of higher and lower sectral radius (meaning magnitude of non-zero effects) are considered in Aendix 9. and the sulement. Hard threshold of 0.0 is alied to final estimates to alleviate noise, all results are averaged over 50 relicates. Table below describes how increase in number T of observed time oints er subect affects the erformance. T FP 0.05(0.0) 0.06(0.0) 0.09(0.0) 0.04(0.0) 0.06(0.0) 0.07(0.0) 0.05(0.0) 0.06(0.0) 0.07(0.0) FN 0.5(0.) 0.05(0.08) 0.0(0.0) 0.0(0.03) 0(0.0) 0.0(0.0) MC 0.8(0.7) 0.89(0.06) 0.9(0.0) 0.95(0.03) 0.94(0.0) 0.93(0) 0.95(0.0) 0.94(0.0) 0.93(0.0) FP (ind) =0, K=0 =30, K=0 =40, K=0 FN (ind) 0.94(0.03) 0.88(0.06) 0.87(0.08) 0.89(0.04) 0.89(0.04) 0.93(0.03) 0.88(0.06) 0.9(0.04) 0.95(0.04) MC (ind) 0.6(0.05) 0.5(0.07) 0.4(0.) 0.3(0.05) 0.4(0.05) 0.9(0.04) 0.4(0.06) 0.(0.05) 0.4(0.06) N ter.6(.3).7(.05).(0.85).5(.08).3(0.97).06(0.5).(.45).3(0.95).09(0.93) Table : Two-stage rocedure erformance for increasing number T of time oints, sectral radius of 0.4 We witness the tendency of denser common comonent estimates, which is manifested in decreasing false negative rate (e.g. from 5 to % for =0) and increasing false ositive rate (from 5 to 9% for =0) within each setting. Meanwhile, the quality of individual comonent estimation is inconsistent, highly deending on the false ositives of common comonent absorbing some of the subect-secific effects due to orthogonality assumtion (see = 40 setting). t shows that for a moderate signal strength (sectral radius of 0.4) the roosed method is more alicable for high-dimensional settings, with number T of time oints not being much higher than the number of variables er subect (maing it a great tool for neuroimaging grou studies). Another asect is reflected in the Table below - the effect of increasing the number K of subects er grou, which would allow us to borrow more strength across the grou for estimation. K FP 0.06(0.0) 0.03(0.0) 0.0(0.0) 0.06(0.0) 0.03(0) 0.0(0) 0.06(0.0) 0.03(0.0) 0.0(0.0) FN 0.05(0.08) 0.06(0.08) 0.05(0.09) 0(0.0) MC 0.89(0.06) 0.9(0.06) 0.95(0.07) 0.94(0.0) 0.97(0) 0.99(0) 0.94(0.0) 0.97(0) 0.99(0) FP (ind) =0, T=80 =30, T=0 =40, T=50 FN (ind) 0.88(0.06) 0.6(0.07) 0.43(0.) 0.89(0.04) 0.43(0.04) 0.9(0.03) 0.9(0.04) 0.43(0.07) 0.6(0.06) MC (ind) 0.5(0.07) 0.5(0.05) 0.63(0.08) 0.4(0.05) 0.63(0.03) 0.8(0.0) 0.(0.05) 0.63(0.05) 0.85(0.05) N ter.7(.05).8(.94) 3.3(.9).3(0.97) 3.(.59) 3.75(.45).3(0.95) 3.(.8) 3.95(.35) Table : Two-stage rocedure erformance for increasing subect grou size K, sectral radius of 0.4 Annex Publishers Volume 3 ssue

6 Journal of Biostatistics and Biometric Alications 6 With increase in the number of subects er grou we see a clear imrovement in the common comonent estimation, which by the structure of our two-stage estimation aroach leads to better estimates for individual effects as well, both of those asects being reflected in resective MC values aroaching. Results for higher sectral radius of 0.6 from Aendix and sulementary materials reaffirm the reference to increasing the number K of subects over obtaining more time oints T er subect. Meanwhile, in the the case of very small sectral radius (0.5, with minimum allowed effect magnitude of 0.), we learn to emhasize collecting higher number T of time oints er subect. Due to smaller signal from true underlying temoral effects being mixed u with the noise, longer time series is necessary for better temoral signal detection (see Aendix ). On the other hand, increasing the number of observed subects rior to obtaining long enough time series may lead to bigger confusion between signal and noise (see sulementary materials). To sum u, the simulation studies demonstrate the reference for increasing the number of subects in a study for moderate to high temoral signal strength, as oosed to increasing the number of observed time oints, which is referred for smaller temoral effect magnitudes. As a side note, the estimation algorithm always converges in under 0 iterations, averaging about three iterations er run. Alication to Resting-State Brain fmr Data for ADHD Study Primary intended alication for the introduced framewor of common structure is resting-state brain fmr time series data in mental disorder studies. We loo at exeriments dealing with evaluation of inter-regional temoral effects within the brain for atients diagnosed with a certain mental disease (e.g., ADHD, Autism, etc), comaring those to a control grou. We exect the atients from the diseased grou (liewise controls) to have similar tendencies in brain temoral dynamics, while also dislaying certain atient-secific features. The idea of shared structure had been used for fmr data before [4,5], but it estimated contemoraneous deendence between brain regions (functional connectivity). Our estimation rocedure attemts to describe temoral dynamics across those regions (effective connectivity). ssues with Estimating Effective Connectivity of the Brain: Attemts at causal modeling of the cross-region relationshis within the brain have been heavily scrutinized in the literature over the recent years. Even though there has been wor done on dynamic causal modeling for estimating effective connectivity of the brain ([9, 0]), the more recent literature is quic to inoint various drawbacs for such aroach. For examle, a review aer on advances and itfalls in resting-state fmr analysis by [] emhasizes the variations in haemodynamic delay across the regions which may introduce bias into any attemt of estimating causality. [3] roceeds to lay out six detailed reasons for caution when inferring causality for fmr analysis, some of them being otential heterogeneity across different exerimental sites and inability to cature causal relations due to neural activity rocesses occuring quicer than the samling rate of fmr measurements. Nevertheless, we will attemt to alleviate some of the issues discussed above with out oint estimation aroach. First of all, the grou lasso directly addresses one of the issues in [3] by caturing the common abstract rocessing structure, such as which regions of the brain influence which other regions, while also allowing for varying strengths of those influences across the atients. As a reminder, it selects a articular relationshi that is common for all the atients, but the exact effect magnitude can differ across the board. We also roceed to select the studies with same reetition times (TR) of fmr measurements, otherwise rising to ointly estimate temoral effects of different lags. Additionally, the individual comonent accounts for subect-secific influences, e.g. sex or handedness. Lastly, in order to avoid the exeriments yielding different regions of interest (RO) for different subects, we mae sure to use a standard unified brain atlas for region assignment across all the atients. Details of ADHD Study Setu: We will consider the resting-state brain fmr time series data for 0 ADHD atients and 0 controls from the ADHD-00 Global Cometition rovided by Python module nilearn. The samle was collected across five exerimental sites, three of which (NYU Child Study Center, Peing University and Radboud University for NeuroMAGE study) shared a TR of.0s, with the other two (Oregon Health and Science University, Kennedy Krieger nstitute) having a longer TR of.5s. As our model aims at inferring temoral effects within the brain, we roceed to exclude the last two studies from consideration in order to have consistent measurement reetition times across all the subects. That leaves us with ADHD subects (all males; age mean ± SD = 3.85 ± 3.83) and controls (all males; age mean ± SD = 3.7 ± 3.7), resectively. All sites reorted the signal to noise ratio of one. The data re-rocessing stes include corrections for delay in slice acquisition and motion, filtering to remove high frequencies, data standardization and detrending []. As mentioned in the discussion above, we roceed to arcellate the brain into 39 regions according to Multi-Subect Dictionary Learning atlas (MSDL), and following further the stes of [] we summarize the signal over those regions via the mean of voxels weighted by gray matter robabilistic segmentation. Both the anatomical locations of the regions and resulting extracted time series can be found on Figures and, with details on scientific names for each of those brain regions contained in Sulementary Table 3 of Aendix. Moreover, Figure 3 demonstrates autocorrelation lots corresonding to time series extracted for one of the regions. Obvious sie at the first time lag of PACF (bottom right lot) serves as a strong AR() signature, which is resent for vast maority of brain regions under consideration. This leads us to believe in VAR() model, emhasized in our simulation studies, being the aroriate tool for this data. Considering the lac of literature on distributional characterization and asymtotical roerties of estimates resulting from grou lasso rocedure, we defer to conducting descritive analysis via stability selection. The concet of stability selection was introduced in [], with its main idea being the use of bootstra, and subsequent accumulation of the results over all bootstraed samles for Annex Publishers Volume 3 ssue

7 7 Journal of Biostatistics and Biometric Alications further analysis. Subsamling leads to controlling the family-wise tye error rate in multile testing for finite samle sizes, which is considerably more imortant for high-dimensional roblems than an asymtotic statement with the number of obser Figure : Brain regions according to MSDL atlas (Sulementary Table 3 for region names) Figure : Extracted time series for 39 brain regions for a single subect Figure 3: Autocorrelation lots for a single region time series: to - time series lot, bottom left - autocorrelation lot (ACF), bottom right - artial autocorrelation lot (PACF) Annex Publishers Volume 3 ssue

8 Journal of Biostatistics and Biometric Alications 8 further analysis. Subsamling leads to controlling the family-wise tye error rate in multile testing for finite samle sizes, which is considerably more imortant for high-dimensional roblems than an asymtotic statement with the number of obser ADHD Study Results and Discussion: Sulementary Table demonstrates both ADHD and control grous having strong autocorrelation effects for each brain region (blue edges), which is to be exected. As it ertains to inter-regional temoral effects, we have a fair amount of those that are resent in both grous (red edges) and ones that are secific to a certain grou (green edges). The magnitudes of detected temoral effects were (mean ± SD) 0:3 ± 0: for autocorrelation effects, 0:07± 0:05 for inter-regional effects. Going from to to bottom, we start with auditory cortex networ (to two regions) and witness left and right auditory cortex influencing each other for controls, while only woring in one direction for ADHD atients. As described in [5], a lot of studies have shown auditory roblems for ADHD-diagnosed atients, which may be reflected in the lac of communication within the auditory cortex networ for their grou on Sulementary Table. The default mode networ (DMN) has been shown to exerience irregularities for ADHD atients, which leads to disrutions in cognitive erformance and resulting lases of attention (one of the main symtoms of the disease). n articular, discovered the atterns of hyeractivation [6], while [7,8] ointed to deficits in its deactivation, which reduces sensitivity to stimulus. Our results artly reflect those ideas by showing more activity in DMN for ADHD grou: on to of autocorrelation effects, it also contains a cross-region temoral effect of right DMN to medial refrontal cortex. Both ventral attention networs, right (RVAN, from right dorsolateral refrontal down to right inferior temoral cortex) and left (LVAN, from left arietal down to left olar frontal lobe), demonstrate lenty of distinct activity atterns for two grous. As mentioned in [6], in ADHD studies for children they found regions of hyoactivation as well as hyeractivation in VAN. Hyoactive regions manifest ADHD-related deficits in detecting and adusting to environmental irregularities, while hyeractive ones underline distractability-one of the most crucial ADHD-symtoms. Meanwhile, in dorsal attention networ (DAN, left and right intraarietal sulcus) we see better communication for controls, which reinforces results of [9,30], both ointing to abnormalities and lac of interactions in DAN as one of characteristics for ADHD atients. Additionally, emhasizes enhanced intraarietal sulcus activation for controls comared to ADHD atients [3]. As for the visual networ (VN, rimary visual cortex, right and left occiital comlex), it was shown to be a discriminative area when comaring ADHD and control grous in [3], with [33] unveiling lower connectivity atterns in visual and occiital cortexes of ADHD subects. n our case, we witness considerably more activity for the controls with a temoral cross-effect for right/left occiital comlexes, and visual cortex influencing both of those regions as well. Annex Publishers Volume 3 ssue

9 9 Journal of Biostatistics and Biometric Alications Figure 4: Directed grah of temoral effects for ADHD atients (to) and controls (bottom). Nodes on the left - brain regions at time t, right - at time t +. Blue edge - region s autocorrelation effect; red - inter-regional effect resent for both atient grous; green - grou-secific inter-regional effect Salience networ (SN, dorsal/ventral anterior cingulate cortex and anterior insular cortex) for controls demonstrates higher endogenous activity (ventral anterior cingulate cortex affecting the other two regions), while also being heavily influenced by the cingulate insular networ (CN, cingulate, right and left insular cortex). According to [6], this networ lays crucial role in such executive rocesses as decision-maing and rocessing information from the external factors, deficiencies in which are very characteristic for ADHD. Additionally, oint to association between attention lases with reduced re-stimulus activity in anterior cingulate regions [7], reinforcing the lac of inter-regional activation of those regions of the SN for ADHD grou (while showing more communcation and also being influenced by members of CN networ for controls) in our study. Left and right suerior temoral sulcus regionsdislay temoral cross-effect between them for the control grou, while only showing a left to right effect for ADHD, reaffirming the results of [6] who claim hyoactivity in temoral regions for ADHD atients. The last networ showing considerable activity is the aforementioned cingular insular networ (CN). Here, ADHD atients show distinct influence of cingulate cortex region on the right insular cortex, which agrees with hyothesis of cingulate cortex hyeractivation for ADHD subects shown in [6]. n the meantime, controls have a temoral cross-effect between right and left insular cortexes, with both of the regions influencing the members of Salience Networ. Referring to the individual comonent estimates that resulted from our two-stage rocedure described, there haened to be no consistent subect-secific effects for either of the study articiants (non were iced in more than 0% of bootstraed samles). This may be attributable in art to the lac of heterogeneity in our dataset (all the subects being teenage boys), and the simulation study for a similar setting (=40; T=50; K=0) showing roensity for false negatives in individual comonent estimates. Conclusion n this wor we resent a regularized oint estimation rocedure for the setting with multile related VAR models being erturbations of a single underlying common VAR model. Even though the idea of oint estimation has been alied to recision matrices in [], and regularized aroach has been used to roduce sarse estimates of VAR models in [6], they were yet to be combined into a oint rocedure for estimating several homogeneous multivariate time series. Such aroach allows one to borrow strength across multile related subects (time series) to roduce more informed and, due to sarsity, more interretable estimates. Primary intended alication is brain fmr time series for mental disorder studies with atients sharing the same mental status (disease Annex Publishers Volume 3 ssue

10 Journal of Biostatistics and Biometric Alications 0 or healthy control). The final estimates are rovided by a two-stage algorithm that breas down the temoral signal into common and individual comonent, uses all subects to ointly estimate a common comonent via grou lasso, and subtracts the common VAR signal to estimate individual comonents via sarse lasso. The erformance on simulated data is shown to be consistent for common comonent across most of the settings, while individual comonentestimation gradually gets better with increase in either the number of subects er grou, or observed time oints er subects (deending on generated strength of temoral signal). While roducing certain results that find confirmations in the literature, as discussed in the ADHD study results section, our methodology is not devoid of oversimlified, at times questionable, assumtions. First of all, by imlementing VAR modeling framewor we are not accounting for ossibility of a non-linear temoral relationshi between the brain regions. Further investigation would need to be done in order to establish whether the relationshi is linear, and if not-how to model the case of non-linearity. Moreover, a simle averaging of a signal across the voxels within a brain region is done in order to get each searate time series in our ADHD study, which might be a severe oversimlification of true underlying signal structure of the brain. At last, while the iterative two-stage rocedure introduced in Section 5.4 converged in ractice (for all the relicates of both simulated and the ADHD study), we didn t exlicitly state the conditions on the time series data in order to show its convergence in theory. Same goes for the oracle roerties of the estimates resulting uon that convergence, settling for the heuristic study results. The most significant extension would be develoing a hyothesis testing framewor for the resented oint estimation rocedure. There is not enough literature on distributional roerties of estimates resulting from grou lasso rocedure, which is necessary for both testing the significance of temoral relationshis in a single grou, and comaring strength of temoral relationshis across different grous. One of the aers addressing the issues of grou-level inference for regularized estimation is Narayan et al. (05), where it is being referred to as Poulation Post Selection nference. t introduces a testing rocedure for grou-level effects that accounts for uncertainties introduced by both the regularization and inter-subect variability. Unfortunately, they don t use oint estimation aroach and grou lasso enalty, estimating brain networs searately with classic lasso, and therefore not alying directly to our rocedure. The other asect in need of further studying is the oint estimation rocedure s ability to deal with VAR models of various lag orders, along with develoing a lag order selection method. Acnowledgement This wor was suorted in art by Graduate Fellowshi and by National Science Foundation [grantdms-63730]. References. Pavlov DY, Pennoc DM (003) A maximum entroy aroach to collaborative filtering in dynamic, sarse, high-dimensional domains. Adv Neural nf Process Syst, USA.. Song S, Zhan Z, Long Z, Zhang J, Yao L (0) Comarative study of SVM methods combined with voxel selection for obect category classification on fmr data. PLoS One 6: e Michailidis G, d'alche-buc F (03) Autoregressive models for gene regulatory networ inference: Sarsity, stability and causality issues. Math Biosci 46: Banbura M, Giannone D, Reichlin L (00) Large Bayesian vector auto regressions. J Al Econom 5: Tibshirani R (996) Regression shrinage and selection via the lasso. J Royal Stats Soc Series B (Methodological) 58: Basu S, Michailidis G (05) Regularized estimation in sarse high-dimensional time series models. Ann Statist 43: Shuai H, Jing L, Liang S, Jieing Y, Adam F, et al. (00) Learning brain connectivity of Alzheimer s disease by sarse inverse covariance estimation. Neuromage 50: Narayan M, Allen G, Tomson S (05) Two samle inference for oulations of grahical models with alications to functional connectivity. Stat ME Liu A, Chen X, Wang ZJ, McKeown MJ (04) Time varying brain connectivity modeling using FMR signals. n Acoustics, Seech and Signal Processing (CASSP). EEE nt conf Guo J, Levina E, Michailidis G, Zhu J (0) Joint estimation of multile grahical models. Biometria 98: -5.. Danaher P, Wang P, Witten DM (03) The oint grahical lasso for inverse covariance estimation across multile classes. J Royal Stats Soc Series B (Statistical Methodology) 76: Cole DM, Smith SM, Becmann CF (00) Advances and itfalls in the analysis and interretation of resting-state FMR data. Front Syst Neurosci 4: Ramsey JD, Hanson SJ, Hanson C, Halcheno YO, Poldrac RA, et al. (00) Six roblems for causal inference from fmr. Neuroimage 49: Belilovsy E, Varoquaux G, Blascho MB (06) Testing for differences in Gaussian grahical models: alications to brain connectivity. n Advances in NPS Chu SH, Lenglet C, Parhi KK (05) Joint brain connectivity estimation from diffusion and functional MR data. SPE Medical maging 943: Luteohl H (005) New introduction to multile time series analysis, Sringer Science & Business Media. 7. Yuan M, Lin Y (006) Model selection and estimation in regression with groued variables. J Royal Stats Soc Series B (Statistical Methodology) 68: Breheny P, Huang J (009) Penalized methods for bi-level variable selection. Stat nterface : Friston KJ, Harrison L, Penny W (003) Dynamic causal modelling. Neuromage 9: Goebel R, Roebroec A, Kim DS, Formisano E (003) nvestigating directed cortical interactions in time-resolved fmr data using vector autoregressive modeling and Granger causality maing. Magn Reson maging : Varoquaux G, Craddoc RC (03) Learning and comaring functional connectomes across subects. Neuromage 80: Annex Publishers Volume 3 ssue

11 Journal of Biostatistics and Biometric Alications. Meinshausen N, Buhlmann P (00) Stability selection. J Royal Stats Soc Series B (Statistical Methodology) 7: Hardle W, Horowitz J, Kreiss JP (003) Bootstra methods for time series. S 7: Buhlmann P, Kunsch HR (999) Bloc length selection in the bootstra for time series. Comut Stat Data Anal 3: Serrallach B, Groß C, Bernhofs V, Engelmann D, Benner J, et al. (06) Neural biomarers for dyslexia, ADHD, and ADD in the auditory cortex of children. Front Neurosci 0: Cortese S, Kelly C, Chabernaud C, Proal E, Di Martino A, et al. (0) Toward systems neuroscience of ADHD: a meta-analysis of 55 fmr studies. Am J Psychiatry 69: Weissman DH, Roberts K, Visscher K, Woldorff M (006) The neural bases of momentary lases in attention. Nat Neurosci 9: Sato JR, Hoexter MQ, Castellanos XF, Rohde LA (0) Abnormal brain connectivity atterns in adults with ADHD: a coherence study. PLoS One 7: e Castellanos FX, Margulies DS, Kelly C, Uddin LQ, Ghaffari, M, et al. (008) Cingulate-recuneus interactions: a new locus of dysfunction in adult attentiondeficit/hyeractivity disorder. Biol Psychiatry 63: Tomasi D, Volow ND (0) Abnormal functional connectivity in children with attentiondeficit/hyeractivity disorder. Biol Psychiatry 7: Hale TS, Booheimer S, McGough JJ, Phillis JM, McCracen JT (007) Atyical brain activation during simle & comlex levels of rocessing in adult ADHD: an fmr study. J Atten Disord : Zhu CZ, Zang YF, Cao QJ, Yan CG, He Y (008) Fisher discriminative analysis of resting-state brain function for attentiondeficit/hyeractivity disorder. Neuroimage 40: Castellanos FX, Proal E (0) Large-scale brain systems in ADHD: beyond the refrontal-striatal model. Trends Cogn Sci 6: 7-6. Submit your next manuscrit to Annex Publishers and benefit from: Easy online submission rocess Raid eer review rocess Online article availability soon after accetance for Publication Oen access: articles available free online More accessibility of the articles to the readers/researchers within the field Better discount on subsequent article submission Submit your manuscrit at htt:// Annex Publishers Volume 3 ssue