Application of Resampling Methods in Microarray Data Analysis

Transcription

1 Application of Resampling Methods in Microarray Data Analysis Tests for two independent samples Oliver Hartmann, Helmut Schäfer Institut für Medizinische Biometrie und Epidemiologie Philipps-Universität Marburg

2 Folie 2 Overview Tests for two independent samples: Westfall & Young step-down resampling (Dudoit et al.) Application to different data sets estimation of false discovery rate (Tusher et al.) Application to different data sets Summary other methods proposed

3 Folie 3 Properties of Gene Expression Data lots of variables (4608) variables (genes) are highly correlated usually very small number of replicates (between 2 and 50 chips) data does not follow normal distribution (many outliers, heavy tails) (lot s of nonsense measures) For all tests: quality-filtered, standardised log-ratios from cdna arrays

4 Folie 4 One typical questions Which genes are up- or down-regulated in cancerous tissue/ between different cancer sub-types? What does the biologist want? a list with candidate genes a feeling on how good the list is They will confirm the results They will add other biological knowledge

5 Folie 5 Westfall & Young step-down Dudoit et al. (2000) based on t-statistic, adjusted p-values estimated by permutation strong control of the FWER takes into account that data is correlated assumptions: all genes have asymptotically the same null distribution (p-values are monotone in the observed t-statistics across genes)

6 Folie 6 W & Y permutation algorithm t (k) ordered by absolute value 1. permute the data (divide into fake control/ treatment group) 2. compute t-statistics for each gene j 3. compute for the k genes ( b) ( b) ( b) ( b), t ( b) 1 ( j) u = t, u = max( u k ( k ) j j+ ) with monotonic constraints! Repeat B-times and calculate: B ( b) ~ I( u t b j * 1 ( j) p( j) = = B )

7 Folie 7 Comparing W&Y to other methods Unadjusted p-values: 1. Wilcoxon rank test 2. estimation of p-value (for t-statistic) via resampling/permutation B ( b) I( t t b j j ) * 1 p j = = B Adjusted p-values: Šidák single-step adjustment ~ p = 1 (1 ) j p j k

8 Folie 8 Assumptions for Šidák assumes independency of tests! provides only weak control of FWER method does not account for correlation structure without independency assumption: ~ p 1 (1 ) j p j for very general conditions alternative (but much worse) would be Bonferroni single step Bonferroni-Holm step down k

9 Folie 9 Description of Datasets two cancer sub-types (mutated gene) 10 vs. 11 (with 2304 genes) 10 vs. 11 (with 4608 genes) 13 vs. 19 ( ) cancer vs. normal tissue 10 vs. 9 ( ) two cancer sub-types (clinical parameter) 14 vs. 18 ( )

10 Folie 10 Mutated Gene: 10 vs. 11 (2.3k)

11 Folie 11 Mutated Gene: 10 vs. 11

12 Folie 12 Mutated Gene: 13 vs. 19

13 Folie 13 Cancer vs. Normal: 10 vs. 9

14 Folie 14 Clinical Parameter: 14 vs. 18

15 Folie 15 How many permutations? Example: Mutated Gene, 13 vs. 19

16 Folie 16 False discovery rate based on Benjamini & Hochberg 1995 V / R = proportion of false discoveries R = no. all rejected null hypotheses V = no. of false rejections FDR = E(V/R) ( FWER) remember: FWER = P(V 1) FDR controlling procedure as in B&H is not applicable (needs independent test statistics) method proposed by Tusher et al. for small sample size

17 Folie 17 Estimation of the FDR Tusher et al. (2001) aim is to estimate the FDR, not to control the FDR at a given error rate does not assume independency assumes all null hypotheses are true (weak control) estimated FDR seems plausible to approximate strongly controlled FDR d-statistic: d x1 x2 = s + s, 2 2 s1 s2 s = n +, 1 n2 0 s 0 > 0

18 Folie 18 Estimation of the FDR d (i) ordered by magnitude The idea is to calculate expected d E,(i) from all possible permutations compare observed d (i) with d E,(i) and call a gene significant if d (i) > d E,(i) + Algorithm: count number of genes with d (i) > d E,(i) + for the setting of interest for all permutations of the data estimator for FDR = median of significant genes from permutations

19 Folie 19 Results from FDR estimation D Genes called sig. FDR, 2.3k, median / 90th percentile FDR, 4.6k, median / 90th percentile FDR, 4.6k, FDR, 4.6k, 10-9 FDR, 4.6k, / 32 % 0 / 0 % 0 % 0 % 0 % / 53 % 0 / 3 % 0 % 0 % 0 % / 60 % 1,1 / 6 % 0 % 0 % 0 % / 63 % 1,4 / 8 % 0,8 % 0 % 0,9 % / 89 % 5 / 16 % 2,9 % 2,8 % 3,8 %

20 Folie 20 Does the statistic matter? Comparing p-values: W & Y vs. Wilcoxon Example: Mutated Gene, 13 vs. 19

21 Folie 21 Example: Mutated Gene, 13 vs. 19 (s 0 = 0.08) Does the statistic matter? Comparing t-statistic vs. d-statistic

22 Folie 22 Summary Westfall & Young works at medium costs for moderate to large sample size better than Wilcoxon/univariate resampling + Šidák Tusher et al. the only one that works for small sample size t-statistic only good for moderate/large sample size d-statistic better for small sample size best?? looking at different statistics may be useful

23 Folie 23 Other methods proposed ANOVA (with resampling, data used without prior standardisation) Newton et al. (has been extended for multiple chips) Grant et al. (PaGE, false-positive rate) Troendle et al. (control of FDP) Other statistics: Scores (Ben-Dor et al.; Park et al.) maximum likelihood (Ideker et al.)...