STATISTICAL DATA ANALYSIS IN EXCEL

Transcription

1 Microrry Center STATISTICAL DATA ANALYSIS IN EXCEL Prt 1 Introduction to Sttistics Dr. Petr Nzrov petr.nzrov@crp-snte.lu Sttisticl dt nlysis in Excel

2 COURSE OVERVIEW Objectives The course: Reminds sttisticl bsics Gives methodologicl tools for reserch Provides prcticl skill for fst dt nlysis 4 sessions (6 hours) Orgniztion Lectures re integrted with prcticl work PLEASE: sk questions. Understnding is extremely importnt for future prts Sttisticl dt nlysis in Excel 2

3 OUTLINE Lecture 1. Reminding Bsics Descriptive sttistics Explortory nlysis Discrete probbility distribution Continues probbility distribution Look for for dt: Sttisticl dt nlysis in Excel 3

4 TABULAR AND GRAPHICAL PRESENTATION Frequency Distribution Frequency Frequency distribution distribution A tbulr tbulr summry summry dt dt showing showing number number (frequency) (frequency) items items in in ech ech severl severl nonoverlpping nonoverlppingclsses. Mrks A B C B A B B A B C Frequency distribution: Mrk Frequency A 3 B 5 C 2 Totl 10 Reltive frequency distribution: Mrk Frequency A 0.3 B 0.5 C 0.2 Totl 1 In MS Excel use following functions: Percent frequency distribution: =COUNTIF(dt,element) to get number elements found in dt re =SUM(dt) to get sum vlues in dt re Mrk Frequency A 30% B 50% C 20% Totl 100% Sttisticl dt nlysis in Excel 4

5 TABULAR AND GRAPHICAL PRESENTATION Exmple: Pncretitis Study The role smoking in etiology pncretitis hs been recognized for mny yers. To provide estimtes quntittive significnce se fctors, hospitl-bsed study ws crried out in estern Msschusetts nd Rhode Islnd between 1975 nd ptients who hd hospitl dischrge dignosis pncretitis were included in this unmtched cse-control study. The control group consisted 217 ptients dmitted for diseses or thn those pncres nd biliry trct. Risk fctor informtion ws obtined from stndrdized interview with ech subject, conducted by trined interviewer. dpted from Chp T. Le, Introductory Biosttistics pncretitis.xls Pncretitis ptients: Smokers Ex-smokers Ex-smokers Smokers Smokers Smokers Ex-smokers Smokers Smokers Smokers Smokers Smokers Ex-smokers Smokers Smokers Ex-smokers Smokers Smokers Ex-smokers Ex-smokers Smokers Ex-smokers Smokers Smokers Never Smokers Ex-smokers Ex-smokers Smokers Ex-smokers Smokers Smokers Ex-smokers Smokers Smokers Smokers Smokers Smokers Ex-smokers Smokers Smokers Smokers Smokers Smokers Smokers Smokers Smokers Smokers Smokers Never Smokers Smokers Smokers Sttisticl dt nlysis in Excel 5

6 TABULAR AND GRAPHICAL PRESENTATION Frequency Distribution Frequency Frequency distribution distribution A tbulr tbulr summry summry dt dt showing showing number number (frequency) (frequency) items items in in ech ech severl severl nonoverlpping nonoverlppingclsses. In MS Excel use following functions: =COUNTIF(dt,element) to get number elements found in dt re =SUM(dt) to get sum vlues in dt re pncretitis.xls Frequency distribution: Smoking Cses Controls Never 2 56 Ex-smokers Smokers Totl Reltive frequency distribution: Smoking Cses Controls Never Ex-smokers Smokers Totl 1 1 Sttisticl dt nlysis in Excel 6

7 TABULAR AND GRAPHICAL PRESENTATION Br nd Pie Chrts pncretitis.xls 80% 80% 70% 70% 60% 60% 50% 50% 40% 40% 30% 30% 20% 20% 10% 10% 0% 0% Never Ex-smokers Smokers Never Ex-smokers Smokers Pncretitis Pncretitis Control Control Pncretitis Pncretitis Never Never Ex-smokers Ex-smokers Smokers Smokers Control Control Never Never Ex-smokers Ex-smokers Smokers Smokers In MS Excel use following steps: Chrt Wizrd Columns Set dt rnge (both columns Percent freq. distribution) Chrt Wizrd Pie Set dt rnge (one columns Percent freq. distribution) Sttisticl dt nlysis in Excel 7

8 TABULAR AND GRAPHICAL PRESENTATION Mice Dt Series Tordf Tordf MG, MG, Bchmnov Bchmnov AA AA Survey clcium & sodium intke Survey clcium & sodium intke nd nd metbolism metbolism with with bone bone nd nd body body composition dt composition dt Project Project symbol: symbol: Tordf3 Tordf3 Accession Accession number: number: MPD:103 MPD:103 mice.xls 790 mice from different strins prmeter Strting ge Ending ge Strting weight Ending weight Weight chnge Bleeding time Ionized C in blood Blood ph Bone minerl density Len tissues weight Ft weight Sttisticl dt nlysis in Excel 8

9 The following re weights in grms for 970 mice: TABULAR AND GRAPHICAL PRESENTATION Histogrm mice.xls Sorted weights show tht vlues re in grms. Let us divide weight into bins bins bins Weight,g Frequency >= More 0 Sttisticl dt nlysis in Excel 9

10 TABULAR AND GRAPHICAL PRESENTATION Histogrm Now, let us use bin-size = 1 grm Bin Frequency More 0 In Excel use following steps: Frequency Weight, g Specify column bins (intervl) upper-limits Tools Dt Anlysis Histrogrm select input dt, bins, nd output (Anlysis ToolPk should be instlled) use Chrt Wizrd Columns to visulize results Sttisticl dt nlysis in Excel 10

11 TABULAR AND GRAPHICAL PRESENTATION Cumultive Frequency Distribution Cumultive Cumultive frequency frequency distribution distribution A tbulr tbulr summry summry quntittive quntittive dt dt showing showing number number items items with with vlues vlues less less thn thn or or equl equl to to upper upper clss clss limit limit ech ech clss. clss. Frequency Weight, g Cumultive reltive frequency Ogive Weight, g Sttisticl dt nlysis in Excel 11

12 TABULAR AND GRAPHICAL PRESENTATION Sctter Plot mice.xls Let us look on mutul dependency Strting nd Ending weights Ending weight In Excel use following steps: Select dt region Use Chrt Wizrd XY (Sctter) Strting weight Sttisticl dt nlysis in Excel 12

13 TABULAR AND GRAPHICAL PRESENTATION Crosstbultion pncretitis.xls Disese Smoking or pncretitis Totl Ex-smokers Never Smokers Totl In Excel use following steps: Dt Pivot Tble nd PivotChrt MS Office list + Pivot Tble Set rnge, including heders dt Select output nd set lyout by drg-nd-dropping nmes into tble Sttisticl dt nlysis in Excel 13

14 NUMERICAL MEASURES Popultion nd Smple Popultion Popultion prmeter prmeter A numericl numericl vlue vlue used used s s summry summry mesure mesure for for popultion popultion (e.g., (e.g., popultion popultion men men µ, µ, vrince vrince σ 2 σ, 2, stndrd stndrd devition devition σ) σ) POPULATION µ men σ 2 vrince N number elements (usully N= ) SAMPLE x m, men s 2 vrince n number elements Smple Smple sttistic sttistic A numericl numericl vlue vlue used used s s summry summry mesure mesure for for smple smple (e.g., (e.g., smple smple men men m, m, smple smple vrince vrince s s 2, 2, nd nd smple smple stndrd stndrd devition devition s) s) mice.xls 790 mice from different strins All existing lbortory Mus musculus ID Strin Sex Strting ge Ending ge Strting weight Ending weight Weight chnge Bleeding time Ionized C in blood Blood ph Bone minerl density Len tissues weight Ft weight 1 129S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m Sttisticl dt nlysis in Excel 14

15 NUMERICAL MEASURES Mesures Loction Men Men A mesure mesure centrl centrl loction loction computed computed by by summing summing dt dt vlues vlues nd nd dividing dividing by by number number observtions. observtions. x = µ = p = m = x i N n x i ( x true) = i n Medin Medin A mesure mesure centrl centrl loction loction provided provided by by vlue vlue in in middle middle when when dt dt re re rrnged rrnged in in scending scending order. order. Weight Mode Mode A mesure mesure loction, loction, defined defined s s vlue vlue tht tht occurs occurs with with gretest gretest frequency. frequency. Mode = 23 Medin = 23.5 Men = 31.7 Sttisticl dt nlysis in Excel 15

16 NUMERICAL MEASURES Mesures Loction mice.xls Histogrm nd p.d.f. pproximtion medinmen mode Femle proportion p f = Density weight, g Bleeding time In Excel use following functions: = AVERAGE(dt) = MEDIAN(dt) = MODE(dt) Density medin = 55 men = 61 mode = N = 760 Bndwidth = Sttisticl dt nlysis in Excel 16

17 NUMERICAL MEASURES Quntiles, Qurtiles nd Percentiles Percentile Percentile A vlue vlue such such tht tht t t lest lest p% p% observtions observtions re re less less thn thn or or equl equl to to this this vlue, vlue, nd nd t t lest lest (100-p)% (100-p)% observtions observtions re re greter greter thn thn or or equl equl to to this this vlue. vlue. The The 50-th 50-th percentile percentile is is medin. medin. Qurtiles Qurtiles The The 25th, 25th, 50th, 50th, nd nd 75th 75th percentiles, percentiles, referred referred to to s s first first qurtile, qurtile, second second qurtile qurtile (medin), (medin), nd nd third third qurtile, qurtile, respectively. respectively. In Excel use following functions: =PERCENTILE(dt,p) Weight Q 1 = 21 Q 2 = 23.5 Q 3 = 39 Sttisticl dt nlysis in Excel 17

18 NUMERICAL MEASURES Mesures Vribility Interqurtile Interqurtile rnge rnge (IQR) (IQR) A mesure mesure vribility, vribility, defined defined to to be be difference difference between between third third nd nd first first qurtiles. qurtiles. Vrince Vrince A mesure mesure vribility vribility bsed bsed on on squred squred devitions devitions dt dt vlues vlues bout bout men. men. Stndrd Stndrd devition devition A mesure mesure vribility vribility computed computed by by tking tking positive positive squre squre root root vrince. vrince. IQR = Q 3 Q 1 popultion smple σ s x i = N ( ) 2 µ 2 2 ( x x) = n 1 i 2 Smple stndrd devition = s = Popultion stndrd devition = σ = 2 s 2 σ Weight IQR = 18 Vrince = St. dev. = 17.9 In Excel use following functions: =VAR(dt), =STDEV(dt) Sttisticl dt nlysis in Excel 18

19 NUMERICAL MEASURES Mesures Vribility Coefficient Coefficient vrition vrition A mesure mesure reltive reltive vribility vribility computed computed by by dividing dividing stndrd stndrd Stndrd devition devition devition by by men. 100 % men. Weight Men CV = 57% Medin Medin bsolute bsolute devition devition (MAD) (MAD) MAD MAD is is robust robust mesure mesure vribility vribility univrite univrite smple smple quntittive quntittive dt. dt. MAD = medin ( x medin( x) ) i Set 1 Set Set 1 Set 2 Men Medin St.dev MAD Sttisticl dt nlysis in Excel 19

20 NUMERICAL MEASURES Mesures Vribility Skewness Skewness A mesure mesure shpe shpe dt dt distribution. distribution. Dt Dt skewed skewed to to left left result result in in negtive negtive skewness; skewness; symmetric symmetric dt dt distribution distribution results results in in zero zero skewness; skewness; nd nd dt dt skewed skewed to to right right result result in in positive positive skewness. skewness. Skewness = n ( )( ) i n 1 n 2 s i x x 3 dpted from Anderson et l Sttistics for Business nd Economics Sttisticl dt nlysis in Excel 20

21 NUMERICAL MEASURES z-score z-score z-score A vlue vlue computed computed by by dividing dividing devition devition bout bout men men (x (x i - i - x) x) by by stndrd stndrd devition devition s. s. A z-score z-score is is referred referred to to s s stndrdized stndrdized vlue vlue nd nd denotes denotes number number stndrd stndrd devitions devitions x x i is i is from from men. men. Chebyshev s orem For For ny ny dt dt set, set, t t lest lest (1 (1 1/z 1/z 2 2 )) dt dt vlues vlues must must be be within within z stndrd devitions from from men, men, where where z ny ny vlue vlue > For ANY distribution: At lest 75 % vlues re within z = 2 stndrd devitions from men At lest 89 % vlues re within z = 3 stndrd devitions from men At lest 94 % vlues re within z = 4 stndrd devitions from men At lest 96% vlues re within z = 5 stndrd devitions from men z i = x i x s Weight z-score Sttisticl dt nlysis in Excel 21

22 NUMERICAL MEASURES Detection Outliers For bell-shped distributions: Approximtely 68 % vlues re within 1 st.dev. from men Approximtely 95 % vlues re within 2 st.dev. from men Almost ll dt points re inside 3 st.dev. from men Outlier Outlier An An unusully unusully smll smll or or unusully unusully lrge lrge dt dt vlue. vlue. For For bell-shped bell-shped distributions distributions dt dt points points with with z >3 z >3 cn cn be be considered considered s s outliers. outliers. Exmple: Gussin distribution Weight z-score Sttisticl dt nlysis in Excel 22

23 NUMERICAL MEASURES Explortion Dt Anlysis Five-number Five-number summry summry An An explortory explortory dt dt nlysis nlysis technique technique tht tht uses uses five five numbers numbers to to summrize summrize dt: dt: smllest smllest vlue, vlue, first first qurtile, qurtile, medin, medin, third third qurtile, qurtile, nd nd lrgest lrgest vlue vlue children.xls Min. Min. : : Q 1 : 1 : Medin: Medin: Q 3 : 3 : Mx. Mx. : : In Excel use: Tool Dt Anlysis Descriptive Sttistics Box Box plot plot A grphicl grphicl summry summry dt dt bsed bsed on on five-number five-number summry summry Min Q 2 Q 1 Box Qplot 3 Mx In Excel use (indirect): Chrt Wizrd Stock Open-high-low-close open Q3 high Q3+1.5*IQR low Q1-1.5*IQR close Q1 1.5 IQR Sttisticl dt nlysis in Excel 23

24 NUMERICAL MEASURES Exmple: Mice Weight Exmple Exmple Build Build box box plot plot for for weights weights mle mle nd nd femle femle mice mice mice.xls 1. Build 5 number summries for mles nd femles Femle Mle Min Q Q Q Mx Combine numbers into following order open Q3 high Q3+min(1.5*(Q3-Q1),Mx) low Q1-mx(1.5*(Q3-Q1),Min) close Q1 Mouse weight In Excel use: Chrt Wizrd Stock Open-high-low-close Put series-in-rows Adjust colors, etc Weight, g Femle Mle Sttisticl dt nlysis in Excel 24

25 NUMERICAL MEASURES Mesure Assocition between 2 Vribles Covrince Covrince A mesure mesure liner liner ssocition ssocition between between two two vribles. vribles. Positive Positive vlues vlues indicte indicte positive positive reltionship; reltionship; negtive negtive vlues vlues indicte indicte negtive negtive reltionship. reltionship. σ xy = popultion ( xi µ x )( yi µ y ) N s xy smple ( x x)( y y) = n 1 i i mice.xls Ending weight vs. Strting weight Ending weight Strting weight In Excel use function: =COVAR(dt) s xy = 39.8 hrd to interpret Sttisticl dt nlysis in Excel 25

26 NUMERICAL MEASURES Mesure Assocition between 2 Vribles Correltion Correltion (Person (Person product product moment moment correltion correltion coefficient) coefficient) A mesure mesure liner liner ssocition ssocition between between two two vribles vribles tht tht tkes tkeson on vlues vlues between between -1-1 nd nd Vlues Vlues ner ner indicte indicte strong strong positive positive liner liner reltionship, reltionship, vlues vlues ner ner -1-1 indicte indicte strong strong negtive negtive liner liner reltionship; reltionship; nd nd vlues vlues ner ner zero zero indicte indicte lck lck liner liner reltionship. reltionship. popultion ( x x)( y y) σ xy i i ρxy = = σ σ σ σ N x y x y r xy smple sxy = = s s x y ( xi x)( yi y) s s ( n 1) x y 60 Ending weight In Excel use function: =CORREL(dt) r xy = Strting weight mice.xls Sttisticl dt nlysis in Excel 26

27 NUMERICAL MEASURES Correltion Coefficient If If we we hve hve only only 2 dt dt points points in in x x nd ndy y dtsets, dtsets, wht wht vlues vlues would would you you expect expect for for correltion correltion b/w b/w xx nd nd y y? Wikipedi Sttisticl dt nlysis in Excel 27

28 DISCRETE PROBABILITY DISTRIBUTION Discrete nd continuous probbility distributions discrete probbility distribution continuous probbility distribution norml probbility distribution Sttisticl dt nlysis in Excel 28

29 RANDOM VARIABLES Rndom Vribles Rndom Rndom vrible vrible A numericl numericl description description outcome outcome n n experiment. experiment. A rndom vrible is lwys numericl mesure. Roll die Number clls to reception per hour Discrete Discrete rndom rndom vrible vrible A rndom rndom vrible vrible tht tht my my ssume ssume eir eir finite finite number number vlues vlues or or n n infinite infinite sequence sequence vlues. vlues. Continuous Continuous rndom rndom vrible vrible A rndom rndom vrible vrible tht tht my my ssume ssume ny ny numericl numericl vlue vlue in in n n intervl intervl or or collection collection intervls. intervls. Time between clls to reception Volume smple in tube Weight, height, blood pressure, etc Sttisticl dt nlysis in Excel 29

30 DISCRETE PROBABILITY DISTRIBUTIONS Discrete Probbility Distribution Probbility Probbility distribution distribution A description description how how probbilities probbilities re re distributed distributed over over vlues vlues rndom rndom vrible. vrible. Probbility Probbility function function A function, function, denoted denoted by by f(x), f(x), tht tht provides provides probbility probbility tht tht xxssumes ssumes prticulr prticulr vlue vlue for for discrete discrete rndom rndom vrible. vrible. Roll die Rndom vrible X: x = 1 x = 2 x = 3 x = 4 x = 5 x = 6 Probbility function f(x) Probbility function f(x) f ( x) 0 f ( x) = 1 Probbility distribution for die roll Probbility distribution for die roll Vrible x Vrible x Probbility function f(x) Probbility function f(x) Number cells under microscope Rndom vrible X: x = 0 x = 1 x = 2 x = 3 P.D. for number cells Probbility distribution for die roll Probbility distribution for die roll Vrible x Vrible x Sttisticl dt nlysis in Excel 30

31 CONTINUOUS PROBABILITY DISTRIBUTIONS Probbility Density Probbility Probbility density density function function A function function used used to to compute compute probbilities probbilities for for continuous continuous rndom rndomvrible. The The re re under under grph grph probbility probbility density density function function over over n n intervl intervl represents represents probbility. probbility Probbility Probbility density density Are =1 x f ( x) = Vrible Vrible x x Sttisticl dt nlysis in Excel 31

32 CONTINUOUS PROBABILITY DISTRIBUTIONS Norml Probbility Distribution Norml Norml probbility probbility distribution distribution A continuous continuous probbility probbility distribution. distribution. Its Its probbility probbility density density function function is is bell bell shped shped nd nd determined determined by by its its men men µ µ nd nd stndrd stndrd devition devition σ. σ. f ( x µ ) 1 2 2σ ( x) = e σ 2π 2 In Excel use function: = NORMDIST(x,m,s,flse) for probbility density function = NORMDIST(x,m,s,true) for cumultive probbility function norml distribution (re from left to x) Sttisticl dt nlysis in Excel 32

33 CONTINUOUS PROBABILITY DISTRIBUTIONS Stndrd Norml Probbility Distribution Stndrd Stndrd norml norml probbility probbility distribution distribution A norml norml distribution distribution with with men men zero zero nd nd stndrd stndrd devition devition one. one. f ( x) = 1 e 2π 2 x 2 z x µ = x σ = zσ + µ In Excel use function: = NORMSDIST(z) Sttisticl dt nlysis in Excel 33

34 CONTINUOUS PROBABILITY DISTRIBUTIONS Dose Selection Exmple Exmple Assume Assume tht tht you you hve hve developed developed n n extremely extremely efficient efficient chemicl chemicl tretment tretment for for glioblstom. glioblstom. During During tests tests on on niml niml models models it it ws ws found found tht tht substnce substnce X, X, which which you you use, use, is is ble ble to to kill kill ll ll tumor tumor cells cells (oreticlly), (oreticlly), but but being being given given t t high high concentrtion concentrtion it it leds leds to to deth deth ptient ptient due due to to intoxiction. intoxiction. As As survived survived cncer cncer cells cells fst fstevolve into into resistnt resistnt form, form, efficiency efficiency tretment tretment is is significntly significntly reduced reduced if if second second course course is is given. given. Therefore Therefore tretment tretment should should be be performed performed in in one one injection. injection. The The experimentl experimentl dt dt suggest suggest tht tht verge verge concentrtion concentrtion needed needed for for positive positive tretment tretment is is 1 µg/kg. µg/kg. The The concentrtion concentrtion needed needed for for effective effective tretment tretment is, is, course, course, rndom rndom vrible. vrible. Being Being presented presented in in log10 log10 scle scle nd nd in in g/kg, g/kg, it it cn cn be be pproximted pproximted by by norml norml rndom rndom vrible vrible with with men men 6 6 nd nd stndrd stndrd devition devition The The 50% 50% lethl lethl dose dose for for humn humn is is µg/kg. µg/kg. And And tests tests on on nimls nimls suggest suggest tht tht in in log10 log10 scle scle it it hs hs norml norml distribution distribution s s well well with with stndrd stndrd devition devition prmeter ug/kg log scle men positive tretment 1-6 std positive tretment x 0.4 men lethl dose std lethl dose x 0.3 Sttisticl dt nlysis in Excel 34

35 CONTINUOUS PROBABILITY DISTRIBUTIONS Dose Selection prmeter ug/kg log scle men positive tretment 1-6 std positive tretment x 0.4 men lethl dose std lethl dose x 0.3 pdf. tretment success pdf. deth due to tretment In Excel use function: = NORMDIST(x,men,std,FALSE) Sttisticl dt nlysis in Excel 35

36 CONTINUOUS PROBABILITY DISTRIBUTIONS Probbility to die from disese = inverse probbility to tret Over-dose nd disese behviors re independent => P( survive ) = P( hel disese ) * P( survive tretment ) Dose Selection probbility recidive diseseprobbility recidive disese probbility to die substnce probbility to die substnce problility to survive In Excel use function: = NORMDIST(x,men,std,TRUE) Sttisticl dt nlysis in Excel 36

37 SAMPLING DISTRIBUTION Smpling distribution smple nd popultion nd ir prmeters centrl limit orem types smpling Sttisticl dt nlysis in Excel 37

38 SAMPLING DISTRIBUTION Prmeters Popultion Popultion prmeter prmeter A numericl numericl vlue vlue used used s s summry summry mesure mesure for for popultion popultion (e.g., (e.g., men men µ, µ, vrince vrince σ 2 σ, 2, stndrd stndrd devition devition σ, σ, proportion proportion π) π) POPULATION µ men σ 2 vrince N number elements (usully N= ) SAMPLE x m, men s 2 vrince n number elements Smple Smple sttistic sttistic A numericl numericl vlue vlue used used s s summry summry mesure mesure for for smple smple (e.g., (e.g., smple smple men men m, m, smple smple vrince vrince s s 2, 2, nd nd smple smple stndrd stndrd devition devition s) s) mice.xls 790 mice from different strins All existing lbortory Mus musculus ID Strin Sex Strting ge Ending ge Strting weight Ending weight Weight chnge Bleeding time Ionized C in blood Blood ph Bone minerl density Len tissues weight Ft weight 1 129S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m Sttisticl dt nlysis in Excel 38

39 SAMPLING DISTRIBUTION Exmple: Mking Rndom Smpling mice.xls 790 mice from different strins ID Strin Sex Strting ge Ending ge Strting weight Ending weight Weight chnge Bleeding time Ionized C in blood Blood ph Bone minerl density Len tissues weight Ft weight 1 129S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m Add column to tble 2. Fill it with =RAND() 3. Sort ll tble by this column 4. Assume tht se mice is popultion with size N=790. Build 3 smples with n=20 5. Clculte m, s for ending weight nd p proportion mles for ech smple Point Point estimtor estimtor The The smple smple sttistic, sttistic, such such s s m, m, s, s, or or p, p, tht tht provides provides point point estimtion estimtion popultion popultion prmeters prmeters µ, µ, σ, σ, π. π. Sttisticl dt nlysis in Excel 39

40 SAMPLING DISTRIBUTION Smpling Distribution Smpling Smpling distribution distribution A probbility probbility distribution distribution consisting consisting ll ll possible possible vlues vlues smple smple sttistic. sttistic. Distribution m Distribution p E(m) = µ Density Density E( p) = π N = Bndwidth = N = Bndwidth = 0.03 σ = m σ n σ p = π ( 1 π ) n Sttisticl dt nlysis in Excel 40

41 SAMPLING DISTRIBUTION Centrl Limit Theorem Centrl Centrl limit limit orem orem In In selecting selecting simple simple rndom rndom smple smple size size n from from popultion, popultion, smpling smpling distribution distribution smple smple men men m cn cn be be pproximted pproximtedby by norml norml distribution distributions s smple smple size size becomes becomes lrge lrge In prctice if smple size is n>30,, norml distribution is good pproximtion for smple men for ny initil distribution. Sttisticl dt nlysis in Excel 41

42 SAMPLING METHODS Strtified Smpling Strtified Strtified rndom rndom smpling smpling A probbility probbility smpling smpling method method in in which which popultion popultion is is first first divided divided into into strt strt nd nd simple simple rndom rndom smple smple is is n n tken tken from from ech ech strtum. strtum. Strt Smple Scientific Institution with 250 coworkers Administrtive Administrtive bord bord people people Reserchers Reserchers people people Engineers Engineers people people Technicins Technicins people people Students Students people people Sttisticl dt nlysis in Excel 42

43 SAMPLING METHODS Cluster Smpling Cluster Cluster smpling smpling A probbility probbility smpling smpling method method in in which which popultion popultion is is first first divided divided into into clusters clusters nd nd n n simple simple rndom rndom smple smple clusters clusters is is tken. tken. Clusters Luxembourg-cmp Luxembourg-cmp Esch-sur-Alzette Esch-sur-Alzette Remich Remich Diekirch Diekirch Smple Smple Mersch Mersch Rednge Rednge etc. Sttisticl dt nlysis in Excel 43

44 SAMPLING METHODS Systemtic Smple Systemtic Systemtic smpling smpling A probbility probbility smpling smpling method method in in which which we we rndomly rndomly select select one one first first kk elements elements nd nd n n select select every every k-th k-lement refter. refter Smple Sttisticl dt nlysis in Excel 44

45 SAMPLING METHODS Convenience Smpling Convenience Convenience smpling smpling A nonprobbility nonprobbilitymethod smpling smpling whereby whereby elements elements re re selected selected for for smple smple on on bsis bsis convenience. convenience. Sttisticl dt nlysis in Excel 45

46 SAMPLING METHODS Judgment Smpling Judgment Judgment smpling smpling A nonprobbility nonprobbilitymethod smpling smpling whereby whereby elements elements re re selected selected for for smple smple bsed bsed on on judgment judgment person person doing doing study. study. Perform selection most confident or most experienced experts. Sttisticl dt nlysis in Excel 46

47 AN EXAMPLE Be Creful with Smpling!!! Spitfire: nlysis dmge Were to put dditionl protection? Sttisticl dt nlysis in Excel 47

48 INTERVAL ESTIMATION Intervl estimtion intervl estimtion popultion men: σ known popultion proportion popultion men: σ unknown Student s distribution estimtion size smple Sttisticl dt nlysis in Excel 48

49 POPULATION AND SAMPLE Prmeters Popultion Popultion prmeter prmeter A numericl numericl vlue vlue used used s s summry summry mesure mesure for for popultion popultion (e.g., (e.g., men men µ, µ, vrince vrince σ 2 σ, 2, stndrd stndrd devition devition σ, σ, proportion proportion π) π) POPULATION µ men σ 2 vrince N number elements (usully N= ) SAMPLE x m, men s 2 vrince n number elements Smple Smple sttistic sttistic A numericl numericl vlue vlue used used s s summry summry mesure mesure for for smple smple (e.g., (e.g., smple smple men men m, m, smple smple vrince vrince s s 2, 2, nd nd smple smple stndrd stndrd devition devition s) s) mice.txt 790 mice from different strins All existing lbortory Mus musculus ID Strin Sex Strting ge Ending ge Strting weight Ending weight Weight chnge Bleeding time Ionized C in blood Blood ph Bone minerl density Len tissues weight Ft weight 1 129S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ f S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m S1/SvImJ m Sttisticl dt nlysis in Excel 49

50 INTERVAL ESTIMATION Intervl Estimtion Intervl Intervl estimte estimte An An estimte estimte popultion popultion prmeter prmeter tht tht provides provides n n intervl intervl believed believed to to contin contin vlue vlue prmeter. prmeter. For For intervl intervl estimtes estimtes in in this this chpter, chpter, it it hs hs form: form: point point estimte estimte ± ± mrgin mrgin error. error. Mrgin Mrgin error error The The ± ± vlue vlue dded dded to to nd nd subtrcted subtrcted from from point point estimte estimte in in order order to to develop develop n n intervl intervl estimte estimte popultion popultion prmeter. prmeter. Density Distribution m µ = m ± merror N = Bndwidth = σσ known known The The condition condition existing existing when when historicl historicl dt dt or or or or informtion informtion provides provides good good vlue vlue for for popultion popultion stndrd stndrd devition devition prior prior to to tking tking smple. smple. The The intervl intervl estimtion estimtion procedure procedure uses uses this this known known vlue vlue о о in in computing computing mrgin mrgin error. error. σσ unknown unknown The The condition condition existing existing when when no no good good bsis bsis exists exists for for estimting estimting popultion popultion stndrd stndrd devition devition prior prior to to tking tking smple. smple. The The intervl intervl estimtion estimtion procedure procedure uses uses smple smple stndrd stndrd devition devition s s in in computing computing mrgin mrgin error. error. Sttisticl dt nlysis in Excel 50

51 INTERVAL ESTIMATION Popultion Men: σ Known Sttisticl dt nlysis in Excel 51

52 INTERVAL ESTIMATION Popultion Men: σ Known Confidence Confidence level level The The confidence confidence ssocited ssocited with with n n intervl intervl estimte. estimte. For For exmple, exmple, if if n n intervl intervl estimtion estimtion procedure procedure provides provides intervls intervls such such tht tht 95% 95% intervls intervls formed formed using using procedure procedure will will include include popultion popultion prmeter, prmeter, intervl intervl estimte estimte is is sid sid to to be be constructed constructed t t 95% 95% confidence confidence level. level. Confidence Confidence intervl intervl Anor Anor nme nme for for n n intervl intervl estimte. estimte. = m ± z µ α / 2 σ n For 95 % confidence α = 0.05, which mens tht in ech til we hve Corresponding z α/2 = 1.96 In Excel use one following functions: = CONFIDENCE(lph, σ, n) = -NORMINV(lph/2,0,1)*σ/SQRT(n) α/2 = 0.95 α/2 = Sttisticl dt nlysis in Excel 52

53 INTERVAL ESTIMATION Popultion Proportion σ p π ( 1 π ) = n π = p ± zα / 2 σ p = p(1 p) n p( 1 p) n if np 5 5 nd n(1-p) 5 Density Smpling distribution for Distribution proportion p π N = Bndwidth = 0.03 pncretitis.txt n= 270 p(never)= sp= E= Prcticl Work Define Define 95% 95% confidence confidence intervl intervl for for never-smoking never-smoking proportion proportion people people coming coming to to hospitl hospitl π π = ± ± % for 95% confidence z = 1.96 Sttisticl dt nlysis in Excel 53

54 INTERVAL ESTIMATION Popultion Proportion: Some Prcticl Aspects π = p ± zα / 2 p(1 p) n 1. The norml distribution is pplicble only when enough dt points re observed. The rule thumb is: np 5 5 nd n(1-p) 5 p(1-p) p(1-p) p p 2. The mximl mrginl error is observed when p= The estimtion smple size cn be obtined: n 2 z / 2 p(1 p) = α 2 E np 5 5 nd n(1-p) 5 Mrginl Error Mrginl Error p p where p is best guess for π or result preliminry study Sttisticl dt nlysis in Excel 54

55 Weight INTERVAL ESTIMATION Popultion Men: σ Unknown Assume tht we hve smple 20 mice nd would like to estimte te n verge size mice in popultion. m = s = 8.84 σ m = σ n s n As we replce σ s, we introduce n dditionl error nd this chnge distribution from z to t (Student) Note: not relistic scle here for illustrtion only Sttisticl dt nlysis in Excel 55

56 INTERVAL ESTIMATION Popultion Men: σ Unknown t-distribution t-distribution A fmily fmily probbility probbility distributions distributions tht tht cn cn be be used used to to develop develop n n intervl intervl estimte estimte popultion popultion men men whenever whenever popultion popultion stndrd stndrd devition devition σσ is is unknown unknown nd nd is is estimted estimted by by smple smple stndrd stndrd devition devition s. s. Degrees Degrees freedom freedom A prmeter prmeter t-distribution. t-distribution. When When ttdistribution distribution is is used used in in computtion computtion n n intervl intervl estimte estimte popultion popultion men, men, pproprite pproprite ttdistribution distribution hs hs n 1 degrees degrees freedom, freedom, where where n is is size size simple simple rndom rndom smple. smple. Sttisticl dt nlysis in Excel 56

57 INTERVAL ESTIMATION Popultion Men: σ Unknown Weight m = s = 8.84 s(m) = 1.98 t = 2.09 m.e. = 4.14 In Excel use: µ = m ± tα ( n 1) / 2 s n α/2 = 0.95 α/2 = = TINV(lph, degree--freedom)!!! Sttisticl dt nlysis in Excel 57

58 INTERVAL ESTIMATION Popultion Men: Prcticl Advices Advice 1 Popultion ( µ = m ± tα / n 1) 2 s n not norml norml symmetric skewed highly skewed ny n n ~ 10 n ~ 30 n 50 Advice 2 if if n >100 >100 you you cn cn use use z-sttistics z-sttistics insted insted t-sttistics t-sttistics (error (error will will be be <1.5%) <1.5%) Sttisticl dt nlysis in Excel 58

59 Sttisticl dt nlysis in Excel 59 INTERVAL ESTIMATION Determining Smple Size Let Let s focus on nor spect: how to select proper number expe s focus on nor spect: how to select proper number experiments. riments.? ), ( ), ( = ± = n E n E n E m σ σ µ / 2 / E z n n z E σ σ α α = = / E z n σ α = n n-dependent prt conf. intervl Effect Smple Size

60 QUESTIONS? Thnk you for your ttention to be continued Sttisticl dt nlysis in Excel 60