Practical Experience in the Analysis of Gene Expression Data

Transcription

1 Workshop Biometrical Analysis of Molecular Markers, Heidelberg, 2001 Practical Experience in the Analysis of Gene Expression Data from Two Data Sets concerning ALL in Children and Patients with Nodules in the Thyroid Glands S. Kropf, Otto von Guericke University, Magdeburg O. Kuß, U. Hattenhorst, S. Burdach, Martin Luther University, Halle M. Eszlinger, K. Krohn, University of Leipzig

2 Contents Introduction Methods applied Data sets Results Summary Kropf et al., Heidelberg, November

3 1. Introduction One of the basic questions in the analysis of gene expression data: Detection of genes that are more or less expressed in some cells than in others: e.g., patients with some disease vs. healthy persons (two-sample problem) affected tissue vs. unaffected ones in the same persons (one-sample problem) Kropf et al., Heidelberg, November

4 Statistical concern: Problem of multiple testing: many hundreds or thousands of genes considered in parallel. In literature and own proposals: methods to control the experimentwise (familywise) type one error. Own experience was based on small samples of lowdimensional arrays (Macro arrays with 588 Genes, each two-fold spotted). Question: Does that also work with high-dimensional arrays? Kropf et al., Heidelberg, November

5 More detailed: Is it statistically feasible two keep the experimentwise error rate? (Can we find significant variables or is the claim to high to be fulfilled with higher dimensions?) Is it technically feasible? on a standard PC? with standard statistical software (SAS, SPSS)? Are there alternative techniques or claims? What about data transformations? Kropf et al., Heidelberg, November

6 2. Methods applied Methods to control the experimentwise type I error: a) Bonferroni-(Holm-)Method α - adjustment: α / (number of genes i) i = 0,1,... per test easy to implement small effect of Holm correction extremely small effective type I errors robustness questionable in extreme tails Kropf et al., Heidelberg, November

7 b) Permutation procedure of Westfall and Young (Westfall and Young, 1993; Dudoit et al., 2000) Basic tests (here t tests) are first carried out with data in the original data, then repeatedly with permuted samples. Each variable is not compared with itself in the permutation process, but the t value of the most significant original variable is compared to that of the most significant variable in the bth permutation sample, the t value of the second original variable (ordered by significance) is compared to the maximum of that for all variables except that which was originally most significant,... Kropf et al., Heidelberg, November

8 Comments: Procedure is distribution-free because of permutation principle. It uses a parametric test as basic element - insofar power dependent on distribution. Procedure is implemented in the SAS procedure PROC MULTTEST for the comparison of independent samples. It is recommended by the working group of Terry Speed (University of California, Berkeley). Kropf et al., Heidelberg, November

9 c) Parametric sequential procedure (Kropf, 2000), two-sample case Based on the assumption of equal variances in all variables (genes). 1. Sort all variables for decreasing values of the variance in the total sample. 2. Carry out unadjusted two-sample t tests in the order given by step 1 until the first non-significant test. Controls the experimentwise Type I error if all variables are normally distributed with equal variance in both groups. The additional assumption of equal variances between all variables is not necessary for the type I error, but for a good power. One-sample version similar (sort for decreasing values of the mean squared difference from 0). Kropf et al., Heidelberg, November

10 d) Nonparametric sequential procedure, two-sample case Also dependent on the assumption of equal variance (shape, interquartile range). 1. Sort variables for decreasing values of the interquartile range of the total sample. 2. Carry out Mann-Whitney-U-Tests in that order until the first non-significant test. Utilizes independence of rank and order statistics. Equal variance of variables important for power, not for type I error. Insofar dependent on distribution. Kropf et al., Heidelberg, November

11 3. Data sets A: acute lymphatic leukemia (ALL) in children (U. Hattenhorst, S. Burdach, O. Kuß, Halle) 11 children with ALL vs. 10 healthy persons, special Affymetrix (R) chip with genes, selection of genes after dropping all rows with empty description field or with a description starting with EST, skewed distributions (across genes or patients), also negative expression values - due to back-ground elimination and normalization process. Kropf et al., Heidelberg, November

12 ALL patients: distribution of expression levels across genes ALL02 ALL01 ALL15 ALL19 ALL22 ALL30 ALL31 ALL32 ALL25 ALL28 ALL29 percentiles Kropf et al., Heidelberg, November

13 Two versions of a transformation: 1. logarithms Problem: negative values shifted: ln(x + 200) 2. cubic root Variances of expression values not really equal for different genes in both versions. Genes with small variances will be supressed in sequential procedure. G G G G G G G group controls ALL mean stddev mean stddev E E selection of seven genes with logarithmic transformation Kropf et al., Heidelberg, November

14 B: patients with nodules in the thyroid glands (Krohn, Eszlinger, Leipzig) 5 patients with hot nodules 5 patients with cold nodules each patient tissue from nodule and from unaffected surrounding genes per tissue two versions of expression levels provided by the Affymetrix software: o LogAvg (pre-processing at logarithmic level) o AvgDiff ( usual version problems with negative values and skewed distributions) cubic root transformation Kropf et al., Heidelberg, November

15 Selection of 10 genes (logarithmic values) var001 var002 var003 var004 var005 var006 var007 var008 var009 var010 group hot nodules cold nodules mean stddev mean stddev E E E Variances not too different! Kropf et al., Heidelberg, November

16 The same genes with cubic root transformation VAR001 VAR002 VAR003 VAR004 VAR005 VAR006 VAR007 VAR008 VAR009 VAR010 group hot nodules cold nodules mean stddev mean stddev E Variances more heterogeneous! Kropf et al., Heidelberg, November

17 4. Results A: ALL data Full set of genes (p = , n 1 = 10, n 2 = 11, α = 0.05): Parametric sequential procedure: Matrix language of SPSS, a few minutes processing time on Pentium III PC, no special problems. #local sign. #Bonf. sign. #sequ.sign. logarithmic transf cubic root (of the 11 above) Westfall/Young: SAS can treat variables at most. Kropf et al., Heidelberg, November

18 Reduced set of genes (p = 9.805) all versions based on logarithmic values: # local sign # Bonferroni sign. 7 # parametric sequ. 10 (2 minutes) # nonparametric sequ. 7 1) # Westfall/Young 12 (30 minutes) 1) with standard SPSS procedures borderline load with large pivot tables Kropf et al., Heidelberg, November

19 Nonparametric sequential procedure for restricted ALL data G(1) G(2) G(3) G(4) G(5) G(6) G(7) G(8) G(9) G(10) G(11) G(12) Monte-Carlo-Sig Mannnifikanz(2-seitig) Whitney -U Signifikanz CASE_LBL GROUP G(1) BM BM BM TS TS CB CB CB CB CB ALL ALL ALL ALL ALL ALL ALL ALL ALL ALL ALL Kropf et al., Heidelberg, November

20 B: hot and cold nodules in thyroid glands ( p = ) both types of nodules together vs. surrounding (n = 5 + 5) one-sample problem, Westfall/Young not available in SAS logarithmic data (Affymetrix) cubic root # local sign # Bonf # parametric sequ. 4 4 (1 in common) # nonparametric sequ. 6 Kropf et al., Heidelberg, November

21 b) hot vs. cold nodules (n 1 = n 2 = 5) a lot of local significances (1.465 with ln, with root, both more than 10 % of genes), but nothing at familywise error level (sequential, Holm, Westfall-Young) Again question: is that claim of familywise error level to hard? Kropf et al., Heidelberg, November

22 Alternative claim: False discovery rate (Benjamini/Hochberg, 1995): expected proportion of falsely rejected null hypotheses among all rejected hypotheses. Recently discussed for array analyses (Tusher, Tibshirani, Chu, 2001, Dudoit et al., ISCB, 2001). Advantage: claim is not higher in high-dimensional arrays than in low-dimensional ones. Problems: In a situation with many non-null genes we can hide also a lot of falsely detected genes. 2. Can it be controlled properly? Benjamini and Hochberg s proposal treats only uncorrelated tests. Kropf et al., Heidelberg, November

23 SAS help text: PROC MULTTEST: FDR Option The FDR option requests adjusted p-values using the method of Benjamini and Hochberg. These p-values do not control the familywise error rate, but they do control the false discovery rate in some cases. ALL data (ln): genes, local sign., 7 Holm, 10/7 sequ., 12 Westfall/Young FDR 398 hot/cold nodules (ln): genes, local, 0 familywise, 3 FDR Kropf et al., Heidelberg, November

24 5. Summary The experimentwise error can be controlled even in extreme high dimensions, but only few significant variables will usually be found; dependent on procedure and transformations. Alternative way: FDR seems sensible. We should give several versions of significance. The combination of standard PC / statistical standard software is on borderline with such high dimensions. There are hard and soft restrictions. Raw data should be transformed, but special way may be dependent on type of arrays and others. Kropf et al., Heidelberg, November

25 References: Benjamini, Y., Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. J.R.Statist. Soc. B 57, Dudoit, S, Yang, Y.H., Callow, M.J., Speed, T.P. (2000). Statistical methods for identifying differentially expressed genes in replicated cdna microarray experiments. Technical Report # 578, Stanford University School of Medicine. Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scand. J. Statist. 6, Kropf, S. (2000). Hochdimensionale multivariate Verfahren in der medizinischen Statistik. Shaker Verlag, Aachen. Tusher, V.G., Tibshirani, R., Chu, G. (2001). Significance analysis of microarrays applied to the ionizing radiation response. PNAS 98, Westfall, P.H. and Young, S.S. (1993). Resampling-based multiple testing. John Wiley & Sons, New York. Kropf et al., Heidelberg, November