VYSOKÉ UČENÍ TECHNICKÉ V BRNĚ

Transcription

1 VYSOKÉ UČENÍ TECHNICKÉ V BRNĚ BRNO UNIVERSITY OF TECHNOLOGY FAKULTA ELEKTROTECHNIKY A KOMUNIKAČNÍCH TECHNOLOGIÍ ÚSTAV BIOMEDICÍNSKÉHO INŽENÝRSTVÍ FACULTY OF ELECTRICAL ENGINEERING AND COMMUNICATION DEPARTMENT OF BIOMEDICAL ENGINEERING ANALYSIS OF RETINAL IMAGE DATA TO SUPPORT GLAUCOMA DIAGNOSIS DIZERTAČNÍ PRÁCE DOCTORAL THESIS AUTOR PRÁCE ING. JAN ODSTRČILÍK AUTHOR BRNO 2014

2 VYSOKÉ UČENÍ TECHNICKÉ V BRNĚ BRNO UNIVERSITY OF TECHNOLOGY FAKULTA ELEKTROTECHNIKY A KOMUNIKAČNÍCH TECHNOLOGIÍ ÚSTAV BIOMEDICÍNSKÉHO INŽENÝRSTVÍ FACULTY OF ELECTRICAL ENGINEERING AND COMMUNICATION DEPARTMENT OF BIOMEDICAL ENGINEERING ANALYSIS OF RETINAL IMAGE DATA TO SUPPORT GLAUCOMA DIAGNOSIS ANALÝZA OBRAZOVÝCH DAT SÍTNICE PRO PODPORU DIAGNOSTIKY GLAUKOMU DIZERTAČNÍ PRÁCE DOCTORAL THESIS AUTOR PRÁCE AUTHOR VEDOUCÍ PRÁCE ACADEMIC ADVISOR Ing. JAN ODSTRČILÍK doc. Ing. RADIM KOLÁŘ, Ph.D. BRNO 2014

3 ABSTRACT Fundus camera s wdely avalable magng devce enablng fast and cheap examnaton of the human retna. Hence, many researchers focus on development of automatc methods towards assessment of varous retnal dseases va fundus mages. Ths dssertaton summarzes recent state-of-the-art n the feld of glaucoma dagnoss usng fundus camera and proposes a novel methodology for assessment of the retnal nerve fber layer (RNFL) va texture analyss. Along wth t, a method for the retnal blood vessel segmentaton s ntroduced as an addtonal valuable contrbuton to the recent state-ofthe-art n the feld of retnal mage processng. Segmentaton of the blood vessels also serves as a necessary step precedng evaluaton of the RNFL va the proposed methodology. In addton, a new publcly avalable hgh-resoluton retnal mage database wth gold standard data s ntroduced as a novel opportunty for other researches to evaluate ther segmentaton algorthms. KEYWORDS Image segmentaton, texture analyss, fundus camera, fundus mage, retna, retnal mage processng, retnal blood vessel, retnal nerve fber layer, ophthalmology, glaucoma.

4 ABSTRAKT Fundus kamera je šroce dostupné zobrazovací zařízení, které umožňuje relatvně rychlé a nenákladné vyšetření zadního segmentu oka sítnce. Z těchto důvodů se mnoho výzkumných pracovšť zaměřuje právě na vývoj automatckých metod dagnostky nemocí sítnce s využtím fundus fotografí. Tato dzertační práce analyzuje současný stav vědeckého poznání v oblast dagnostky glaukomu s využtím fundus kamery a navrhuje novou metodku hodnocení vrstvy nervových vláken (VNV) na sítnc pomocí texturní analýzy. Spolu s touto metodkou je navržena metoda segmentace cévního řečště sítnce, jakožto další hodnotný příspěvek k současnému stavu řešené problematky. Segmentace cévního řečště rovněž slouží jako nezbytný krok předcházející analýzu VNV. Vedle toho práce publkuje novou volně dostupnou databáz snímků sítnce se zlatým standardy pro účely hodnocení automatckých metod segmentace cévního řečště. KLÍČOVÁ SLOVA Segmentace obrazu, texturní analýza, fundus kamera, snímky sítnce, retna, zpracování snímků sítnce, retnální cévní řečště, vrstva nervových vláken, oftalmologe, glaukom.

5 ODSTRČILÍK, J. Analyss of retnal mage data to support glaucoma dagnoss. Brno: Brno Unversty of Technology, Faculty of Electrcal Engneerng and Communcaton, p. Academc advsor: doc. Ing. Radm Kolář, Ph.D.

6 PROHLÁŠENÍ Prohlašuj, že svou dsertační prác na téma Analýza obrazových dat sítnce pro podporu dagnostky glaukomu jsem vypracoval samostatně pod vedením vedoucího dsertační práce a s použtím odborné lteratury a dalších nformačních zdrojů, které jsou všechny ctovány v prác a uvedeny v seznamu lteratury na konc práce. Jako autor uvedené dsertační práce dále prohlašuj, že v souvslost s vytvořením této dsertační práce jsem neporušl autorská práva třetích osob, zejména jsem nezasáhl nedovoleným způsobem do czích autorských práv osobnostních a jsem s plně vědom následků porušení ustanovení 11 a následujících autorského zákona č. 121/2000 Sb., včetně možných trestněprávních důsledků vyplývajících z ustanovení část druhé, hlavy VI. díl 4 Trestního zákoníku č. 40/2009 Sb. V Brně dne.... (podps autora)

7 ACKNOWLEDGEMENT I would lke to thank all the people who supported me durng my doctoral studes. Especally, I would lke to thank to my supervsor doc. Ing. Radm Kolář, Ph.D. for hs professonal mentorng, contrbutng suggestons, deas, and great encouragement. Furthermore, I wsh to thank to prof. Ing. Jří Jan, CSc. for hs nterest, valuable support, and hghly constructve and suggestve dscussons durng my studes. I also thank to prof. Ing. Ivo Provazník, Ph.D., head of the Department of Bomedcal Engneerng, FEEC, BUT for provdng excellent condtons for the study and constantly supportng my research. Next acknowledgement s dedcated to foregn colleagues from the Unversty of Erlangen, Erlangen-Nuremberg, Germany for ther assstance durng measurements and valuable cooperaton n the research. Partcularly, I acknowledge cooperaton wth Dr. Ralf-Peter Tornow, Dr. Robert Laemmer, and MUDr. Tomáš Kuběna who supported measurements and data acquston. I also acknowledge nsttutonal support gven by several research projects, namely DAR (research center Data Algorthms Decson Makng) n the years (project no. 1M0572), two blateral Czech-German grants n the years and (project no. D10-CZ16/09-10, 7AMB12DE002), and FNUSA-ICRC (CZ.1.05/1.1.00/ ) n the years Fnally, specal thanks also belong to my famly, my parents, grlfrend, and frends for ther great patence, love, generous support, and nspraton.

8 TABLE OF CONTENTS 1 INTRODUCTION RETINAL BLOOD VESSEL SEGMENTATION Background State-of-the-art Fundus mage databases for testng of the blood vessel segmentaton algorthms DRIVE STARE HRF Methodology Illumnaton correcton and contrast equalzaton Two-dmensonal matched flterng Thresholdng and postprocessng Results and dscusson Evaluaton methodology Evaluaton of the method usng HRF database Evaluaton of the method usng DRIVE and STARE databases Comparson wth other methods Notes about method mplementaton Concluson ANALYSIS OF FUNDUS IMAGES FOR RETINAL NERVE FIBER LAYER ASSESSMENT Background State-of-the-art Expermental mage database Methodology Data preprocessng Preprocessng of fundus mages Preprocessng of OCT data Fundus-OCT mage regstraton Texture analyss Gaussan Markov random felds Local bnary patterns Pyramdal decomposton Feature selecton and regresson Background of feature selecton... 65

9 Flter methods Wrapper methods Regresson models Results and dscusson Evaluaton methodology Evaluaton of the method va cross-valdaton Complete feature set Flter CFS Flter mrmr Wrapper SFS Wrapper SBS Evaluaton of the method usng crcular scan patterns Notes about method mplementaton Concluson OVERALL CONCLUSION REFERENCES ABBREVIATIONS APPENDIX A HISTOGRAM FEATURES FOR LOCAL BINARY PATTERNS

10 LIST OF FIGURES Fgure 1. A standard fundus mage depctng typcal dagnostcally mportant retnal structures Fgure 2. A fundus mage "01_test.bmp" from a tran set n the DRIVE database: a) an orgnal RGB mage, b) correspondng manual segmentaton of the 1 st and c) 2 nd human observer Fgure 3. A fundus mage "m0001.ppm" from the STARE database: a) an orgnal RGB mage, b) correspondng manual segmentaton of the 1 st and c) 2 nd human observer Fgure 4. Examples of fundus mages from the HRF database: a) mage "06_h.jpg" from the healthy group, b) mage "04_dr.jpg" from the DR group, and c) mage "14_g.jpg" from the group of glaucomatous mages wth correspondng hand labeled gold standard segmentatons Fgure 5. Flowchart of the proposed blood vessel segmentaton approach; where G(,j) green channel of the nput RGB mage, I(,j) - preprocessed mage, h k,(x,y) - convoluton masks (k=0,1,,4 and =0,15,,165 ), MFR k,(,j) - matched flter responses, O(,j) - resultng (fused) parametrc mage, O T(,j) - thresholded parametrc mage, O F(,j) fnally cleaned mage Fgure 6. Partcular colour channels of the mage 06_h.jpg from the HRF database Fgure 7. Preprocessng of the nput mage: a) green channel of the orgnal mage 05_dr.jpg from the HRF database, b) correspondng corrected mage, and c) two-dmensonal B-splne functon Fgure 8. Depcton of averaged blood vessel cross sectonal profles (at the top) expanded nto the correspondng 2D kernels (at the bottom) classfed nto fve classes of dfferent wdths: from the narrowest a) to the wdest e) blood vessel profle; ndces m, n stay for spatal coordnates of partcular matrces.. 33 Fgure 9. Depcton of the blood vessel segmentaton results: a) resultng parametrc mage O(,j) composed from the partcular parametrc mages selectng the maxmum response for each pxel, b) thresholded mage OT(,j) obtaned by the proposed thresholdng algorthm, and c) morphologcally cleaned bnary mage OF(,j) (the mages correspond to the fundus mage n Fgure 4a) Fgure 10. Resultng blood vessel segmentatons (top par of each subfgure) for a) healthy, b) DR, and c) glaucomatous group wth correspondng gold standard hand labeled segmentatons (bottom par of each subfgure). The left and rght hand sde of each subfgure represents mages that acheved maxmum and mnmum value of ACC wthn the partcular group, respectvely Fgure 11. ROC curves plotted for each mage separately concernng mages of a) healthy, b) DR, c) glaucomatous retnas from the HRF database, d) mages from the DRIVE database, and e) mages from the STARE database; ROC charts are zoomed to the top left corner for better vsual dfferentaton between partcular curves Fgure 12. Colour labelng of the blood vessel pxels accordng to detected wdth class n the colour spectral scale: startng from the red for the narrowest class contnung to blue for the wdest class; top part: an overall colour labelng of blood vessel tree segmentaton result, bottom part: detals of colour labelng of the blood vessel pxels accordng to vessel dameter

11 Fgure 13. Comparson of the best (left) and the worst (rght) results measured by ACC usng the DRIVE database; at the top: results of the proposed method, at the bottom: gold standard segmentatons Fgure 14. a) Secton of the green component of the mage 01_test.tf n the test set of the DRIVE database wth correspondng gold standard mages labeled by b) 1 st observer and c) 2 nd observer as a proof of dffculty of human observer to decde whether there s a vessel or not Fgure 15. Major structures of the ONH for a) healthy eye, b) glaucomatous eye Fgure 16. Depcton of the RNFL strated pattern n the area around the ONH for a) healthy eye, b) glaucomatous eye wth dstnctve wedge-shaped RNFL loss Fgure 17. An example of orgnal RGB fundus mage of the healthy rght eye and partcular colour channels. In standard fundus mage, the red (R) channel appears oversaturated, whle the green (G) and the blue (B) channel show the blood vessels and retnal nerve fber layer well contrasted Fgure 18. An example of OCT volume and crcular scans. a) SLO mage (left) wth the volume scan pattern allocated by the green lnes and one B-scan (rght) measured at the poston depcted by the blue lne n SLO; b) SLO mage (left) wth the crcular scan pattern defned by the blue crcle and the B-scan (rght) measured along ths crcle n drecton gven by the arrow Fgure 19. Schematc dagram of the proposed methodology for the RNFL assessment Fgure 20. The ONH surroundngs of the preprocessed fundus mage of a normal subject and ROIs depcted by the red colour. Magnfed examples of eght ROIs relatng to the dfferent postons n fundus mage are shown on the fgure s sdes Fgure 21. OCT volume data: a) the RNFL thckness map mapped on the SLO mage; the color bar on the top shows values of the RNFL thckness measured n m, b) one B scan acqured at the poston marked by the black lne n a); the RNFL s segmented between the red and the green curves Fgure 22. A ffth order symmetrc rotaton nvarant neghborhood structure Fgure 23. Schematc dagram of the fnal feature vector Fgure 24. Cross-valdaton results of partcular models usng complete feature set - computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram Fgure 25. Cross-valdaton results for partcular models usng complete feature set - RMSEP computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram Fgure 26. Cross-valdaton results of partcular models usng the flter-based CFS approach - computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram

12 Fgure 27. Cross-valdaton results for partcular models usng the flter-based CFS approach - RMSEP computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram Fgure 28. Cross-valdaton results of partcular models usng flter-based mrmr approach - computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram Fgure 29. Cross-valdaton results for partcular models usng the flter-based mrmr approach - RMSEP computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram Fgure 30. Cross-valdaton results of partcular models usng the wrapper-based SFS search strategy - computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram Fgure 31. Cross-valdaton results for partcular models usng the wrapper-based SFS search strategy - RMSEP computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram Fgure 32. Cross-valdaton results of partcular models usng the wrapper-based SBS search strategy - computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram Fgure 33. Cross-valdaton results for partcular models usng the wrapper-based SBS search strategy - RMSEP computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram Fgure 34. Relaton between the -SVR predcted output and the RNFL thckness for (a) complete feature set and (b) feature subset dentfed va the wrapper-based SFS approach. The model output was computed for each of the 354 ROIs Fgure 35. Images of crcular scans of the normal subject no. 1 and correspondng profles: a) orgnal GB fundus mage wth ntensty profle, b) model predcted output wth correspondng profle, c) SLO mage wth crcular scan pattern and the RNFL thckness profle. Red curves represent polynomal approxmaton of each profle. The red arrow ndcates drecton of scannng Fgure 36. Images of crcular scans of the normal subject no. 6 and correspondng profles: a) orgnal GB fundus mage wth ntensty profle, b) model predcted output wth correspondng profle, c) SLO mage wth crcular scan pattern and the RNFL thckness profle. Red curves represent polynomal approxmaton of each profle. The red arrow ndcates drecton of scannng Fgure 37. Images of crcular scans of the glaucomatous subject no. 1 and correspondng profles: a) orgnal GB fundus mage wth ntensty profle, b) model predcted output wth correspondng profle, c) SLO mage wth crcular scan pattern and the RNFL thckness profle. Red curves represent polynomal approxmaton of each profle. The red arrow ndcates drecton of scannng. The RNFL los can be seen approx. at the angular poston of 270-degrees

13 Fgure 38. Images of crcular scans of the glaucomatous subject no. 4 and correspondng profles: a) orgnal GB fundus mage wth ntensty profle, b) model predcted output wth correspondng profle, c) SLO mage wth crcular scan pattern and the RNFL thckness profle. Red curves represent polynomal approxmaton of each profle. The red arrow ndcates drecton of scannng. The RNFL los can be seen approx. at the angular poston of 300-degrees

14 LIST OF TABLES Table 1. Performance evaluaton on the healthy group (HRF); mnmum and maxmum values of each parameter are boldfaced Table 2. Performance evaluaton on the DR group (HRF); mnmum and maxmum values of each parameter are boldfaced Table 3. Performance evaluaton on the glaucomatous group (HRF); mnmum and maxmum values of each parameter are boldfaced Table 4. Performance evaluaton on DRIVE and STARE databases; mnmum and maxmum values of each parameter are boldfaced Table 5. Comparson of the 2 nd and the 1 st human observer segmentatons on the DRIVE database; mnmum and maxmum values of each parameter are boldfaced Table 6. Comparson of the proposed method wth other vessel segmentaton algorthms evaluaton on DRIVE and STARE databases; mnmum and maxmum values of each parameter are boldfaced Table 7. Evaluaton of the proposed method usng the HRF database n comparson wth DRIVE and STARE databases; mnmum and maxmum values of each parameter are boldfaced Table 8. The most popular kernel functons used n SVM/SVR Table 9. An overvew of approaches used for evaluaton of the proposed methodology Table 10. Averaged cross-valdaton results of partcular regresson models usng complete feature set. 78 Table 11. Averaged cross-valdaton results of partcular regresson models usng CFS Table 12. Averaged cross-valdaton results of partcular regresson models usng the flter-based mrmr approach Table 13. Averaged cross-valdaton results of partcular regresson models usng the wrapper-based SFS search strategy Table 14. Averaged cross-valdaton results of partcular regresson models usng the wrapper-based SBS search strategy Table 15. Evaluaton of the method on mages of normal subjects. The values n brackets deal wth approxmated profles (the red curves n Fgure 35 - Fgure 36). The values are computed for the non-vessel locatons only. Mnmum and maxmum values are boldfaced n each column. Hghlghted rows denote the mages that are presented n Fgure 35 - Fgure 36 (for both mages, the wrapper SFS results are dsplayed) Table 16. Evaluaton of the method on mages of glaucomatous subjects. The values n brackets deal wth approxmated profles (the red curves n Fgure 37 - Fgure 38). The values are computed for the non-vessel locatons only. Mnmum and maxmum values are boldfaced n each column. Hghlghted rows denote the

15 mages that are presented n Fgure 37 - Fgure 38 (for both mages, the wrapper SFS results are dsplayed) Table 17. Features computed from one-dmensonal grey-level hstogram Table 18. Features computed from two-dmensonal jont hstogram

16 1 INTRODUCTION The eye s a part of the human body that s used for recevng vsual nformaton from the envronment va lght senstve cells n the retna. The retna s a part of the posteror segment of the eye called fundus; see an example of fundus mage n Fgure 1. Retnal tssue conssts of ten retnal layers [32]. The nner lmtng membrane (ILM) s a layer servng as a boundary between the retna and the vtreous of the eye. On the other sde, there s the retnal pgment epthelum (RPE) layer protectng the retna from excessve lght. Above the RPE, there s a photoreceptor layer consstng of rods and cones, whch detect ncomng sgnal. The lght nformaton from ndvdual photoreceptors s processed by a varety of cells n dfferent retnal layers and fnally read out va ganglon cells (neurons). Axons of these neurons form the retnal nerve fber layer (RNFL) and they ext the retna as a bundle n locaton called the optc nerve head (ONH) or the optc dsc (OD). The vsual nformaton s then delvered va the optc nerve nto the bran. Then, the bran nterprets ncomng nformaton as an mage. The blood vessels enter the retna through the ONH provdng essental perfuson and nutrton for the retnal tssue. Roughly, n the center of retna temporarly from the ONH, there s the macula, small, very senstve, and hghly pgmented area, responsble for detaled central photopc vson. The fovea, also called as yellow spot, contans the largest concentraton of cones and s located n the near center of the macular regon (Fgure 1). Fgure 1. A standard fundus mage depctng typcal dagnostcally mportant retnal structures

17 A posteror segment of the eye s the only one nternal part of the human body that can be used for nonnvasve examnaton of the vascular system. Images of ocular fundus (fundus mages) can help n dagnoss and treatment of many dseases ncludng varous retnopathes, ophthalmc pathologes, glaucoma, and even systemc dseases, such as dabetes, hypertenson or arteroscleross [62]. There are several magng modaltes based on dfferent physcal prncples that can be used for examnaton of the retna [24]. The oldest dagnostc equpment s probably classc analogous ophthalmoscope enablng clncans to examne retnal structures vsually. Nevertheless, ths can provde only a realstc look on the retna wthout any data storage. Ablty to archve dagnostc mages has come wth the frst fundus camera devce (n eghtes of the 19 th century) [165], whch has enabled recordng of the lght reflected from the retna usng black and whte photography. Wth the drve of nformaton technology, development of ophthalmc magng has regstered consderable progress. Nowadays, dgtal magng usng fundus camera s wdely consdered as an ntegral part of medcal examnaton n ophthalmology. Ths enables not only a vsual nspecton of retnal surface, but also archvng and computer aded dagnostc of the acqured data. Except fundus camera, there are also other magng devces n ophthalmology provdng two-dmensonal (2D) or even three-dmensonal (3D) data acquston. Especally, devces enablng 3D acquston of retnal topography and retnal thckness have become of nterest. Important representatves of ths category are devces based on a scannng laser ophthalmoscopy (SLO), whch was frst ntroduced n eghtes of the 20 th century [174]. SLO prncple was later utlzed as a bass for further equpment enablng 2D magng of retnal surface for specfc applcatons, e.g. for the blood flow Doppler magng [105] and angography of the retnal vasculature [45]. Further, n 1987, confocal scannng laser ophthalmoscope (CSLO) was ntroduced as an extenson to SLO [175]. Ths devce enables 3D acquston of retnal structures usng confocal optcs. Probably the best known and the most wdely used CSLO system s the Hedelberg Retnal Tomograph (HRT), developed by Hedelberg Engneerng (Hedelberg, Germany). The HRT system s used especally for 3D magng of the ONH and the macula. By ths system, the retnal structures can be maged to a depth of 3.5 mm, but t depends on utlzed wavelength manly, optcal propertes of the eye and even other acquston factors [24]

18 Another magng modalty, fallng nto the category of devces enablng 3D acquston of retnal volume, s a scannng laser polarmetry (SLP). It s based on CSLO prncple, further supplemented by measurement of polarzed lght reflected from retnal surface [24]. There s one commercal devce usng ths prncple n ophthalmology GDx (Laser Dagnostc Technologes, Inc., San Dego, Calforna, USA). Ths devce s ntended especally for analyss of RNFL thckness utlzng an ablty of ganglon cells to reflect the lght wth dfferent polarzaton [177]. Today, optcal coherence tomography (OCT) s utlzed rather than GDx devce for the RNFL thckness evaluaton. Ths modalty was frst ntroduced for ophthalmology n 1986 [33]. It enables magng of 3D retnal surface usng near-nfrared lght beam ( nm) focused on dfferent retnal layers. Analogously to ultrasound magng, the OCT devce s able to measure axal sgnal (A-scans) as well as slces of retnal volume (Bscans). However, OCT acheves much better spatal resoluton (~10 µm or less) than a typcal ultrasound magng system (~100 µm) [27]. Current ophthalmc OCT systems usually combne two magng modaltes nto one sngle devce SLO and OCT (e.g. Spectrals, Hedelberg Engneerng, Hedelberg, Germany). The retna can be affected due to varous eye and even complex dseases such as hypertenson or dabetes melltus [62]. One of the common retnal dseases s dabetc retnopathy resultng n pathologcal changes of the retnal blood vessel system [32]. Furthermore, macular degeneraton, glaucoma and other retnal dseases affectng the macula, nerve fbers, and even vascular structures occur qute frequently [62]. Many of these dseases can result n partcular sght loss or even permanent blndness. Partcularly, glaucoma s one of the most common causes of permanent blndness worldwde. Glaucoma results n retnal changes, especally n regon of the ONH: an enlargement of the ONH excavaton, the ONH hemorrhages, thnnng of the neuroretnal rm, asymmetry of the cup between left and rght eye, and progressve RNFL atrophy causng decrease of the layer's thckness [39]. Deteroraton of the RNFL results n patent s vsual feld loss that s permanent and unfortunately cannot be reversed. The study [142] shows an avalable data of glaucoma occurrence n 2005 and presents a model predctng expanson of the glaucoma dsease untl the year Accordng to ths study, there wll be about 79.6 mllon people wth glaucoma n 2020 worldwde. Whereas, t was 66.8 mllon people n It s rather obvous that numbers of people wth glaucoma are constantly rsng. Glaucoma symptoms appear many years before the patents are able to observe

19 any changes n the vsual feld of ther eye. It s therefore extremely desrable to set up screenng programs for early glaucoma detecton to be able to start an approprate treatment as soon as possble. Only an mmedate treatment can suffcently allevate effects of the dsease and stop progresson of the RNFL atrophy. Today, glaucoma dagnoss s based on evaluaton of the ONH morphology and the RNFL thckness n the perpapllary area (around the ONH). Many ophthalmc clncs around the world stll use classc ophthalmoscope or fundus camera to evaluate the retna only vsually. Nevertheless, such a qualtatve dagnostc s rather tme consumng and barely reproducble. Especally for screenng purposes, t s hghly requested to enable objectve evaluaton and make dagnostc procedures cost-effectve and easly reproducble [176]. Above-mentoned magng modaltes for glaucoma dagnoss have usually an ablty to evaluate measured data objectvely and automatcally va a commercal software. For example, HRT (HRT II, HRT III) can evaluate topography of the ONH and the macula. GDx and OCT can be used for measurement of the RNFL thckness. However, these "h-tech" devces are rather expensve and stll not generally avalable for many ophthalmc clncs around the world. Hence, fundus camera s stll wdely consdered as a fundamental dagnostc devce. In contrast wth OCT, HRT, and GDx, acquston procedure by fundus camera s much faster, cheaper and generally more bearable for patents. Because of that, there s growng trend to perform screenng of glaucoma usng dgtal fundus cameras, whch are commonly utlzed n many ophthalmc facltes around the world. Snce the technology mproves constantly, modern fundus cameras can be used for acquston of hgh qualty mages that provde relable nformaton about dagnostcally mportant retnal structures. Thanks to ths aspect, many research groups as well as clncal departments focus on research n analyss of fundus mages amed to support an automatc processng of fundus mage data. Nevertheless, recent state-of-the-art shows knowledge nconsstency n utlzaton of fundus cameras for automatc dagnoss of retnal dseases, partcularly of glaucoma, varous retnopathes, and vascular problems. Ths dssertaton summarzes a recent state n the feld of glaucoma dagnoss based on utlzaton of dgtal fundus cameras and proposes a novel methodology for assessment of the RNFL va texture analyss. Along wth t, a method for the blood vessel segmentaton n fundus mages s ntroduced as a valuable contrbuton to the recent stateof-the-art n the feld of retnal mage processng. Segmentaton of the blood vessels also

20 serves as a necessary step precedng evaluaton of the RNFL va the proposed methodology. Besdes that, a new publcly avalable hgh-resoluton retnal mage database wth gold standard data s ntroduced as a novel opportunty for other researchers to evaluate ther blood vessel segmentaton methods. The dssertaton s dvded nto two parts. The frst part s focused on the retnal blood vessel segmentaton. In ths part, the state-of-the-art n ths feld s dscussed and accordng to that, a method for accurate blood vessel segmentaton together wth a new retnal database s proposed. The second part s orented towards texture analyss of the retnal nerve fber layer. Accordng to summarzaton of the state-of-the-art n glaucoma dagnoss, a novel approach for evaluaton of the RNFL pattern n fundus mages s ntroduced. At the end of the dssertaton, overall concluson s provded along wth dscusson of possbltes for further research

21 PART I 2 RETINAL BLOOD VESSEL SEGMENTATION 2.1 Background Ths part s focused on automatc retnal blood vessel segmentaton. Ths topc has been taken nto account frst n early eghtes of the 20 th century [160]. Many publcatons concernng ths topc have been publshed snce ths tme. Although many scentfc works have been already presented, there are stll sgnfcant ssues that need to be solved n ths feld of retnal mage processng. Segmentaton of the retnal blood vessels can be mportant from many ponts of vew. In fact, precsely and accurately segmented blood vessel tree s needed n many applcatons focused on analyss of fundus mages. For example, t can help to fnd dfferent pathologes affectng the retnal vascular structures due to the dabetc retnopathy (neovascular nets, hemorrhages, mcroaneurysms) [40], [109]. Moreover, automatc retnal vessel segmentaton algorthms can be useful n evaluaton of other dseases, such as arterolar narrowng and vessel tortuosty due to the retnopathy of prematurty [61] and hypertensve retnopathy [51], or even glaucoma [169], [173]. Furthermore, vessel dameter, bfurcatons and crossovers can be effectvely measured on the segmented blood vessel tree n order to test for other cardovascular dseases [22], [78], [172]. A precse and accurate detecton of the vascular tree n fundus mages can provde several useful features for a dagnoss of varous retnal dseases. However, retnal blood vessel segmentaton can have a consderable mpact on other applcatons of retnal mage processng as well, partcularly when used as a preprocessng step for hgher-level mage analyss. For example, an accurate detecton of the blood vessel tree can be useful n regsterng longtudnal tme-seres fundus mages [109], locatng the ONH [182] or the fovea (the macula) [114], or n the feld of bometrc dentfcaton [34], [57]

22 Ths dssertaton ntroduces a novel and precse methodology for accurate retnal vessel tree segmentaton at a wde range of blood vessel szes n hgh-resoluton colour fundus mages. The method s based on matched flterng n combnaton wth mnmum error thresholdng technque. Earler verson of ths segmentaton technque has been developed n frame of the author's master thess [117]. The ntal work has been later extended durng the doctoral studes and publshed as several conference papers [17], [74], [119], [120], [121], [122], and as a part of a journal paper [75]. The latest state of the retnal blood vessel segmentaton approach s presented n ths dssertaton and t s completely publshed as a journal paper [123] as well. In comparson wth the prevous states, the latest publcaton presents the method that uses fve dfferent matched flters desgned accordng to the typcal blood vessel ntensty profles for dfferent vessel wdths and uses more precse thresholdng algorthm. Evaluaton of the method s performance s newly carred out va a new fundus mage database and comparson wth other approaches s presented. Nowadays, all authors compare ther vessel segmentaton results to each other usng well-known retnal mage databases such as DRIVE (Dgtal Retnal Images for Vessel Extracton) [156] or STARE (STructured Analyss of the REtna) [63]. Unfortunately, these databases contan only outdated low-resoluton retnal mages, whch are napproprate for the evaluaton of methods for detectng fne blood vessel structures. Thus, as the man contrbuton to the feld of retnal vessel segmentaton, a new publcly avalable hgh-resoluton fundus mage database s ntroduced n a frame of ths dssertaton. The database contans mages of healthy and pathologcal retnas wth correspondng manually labeled mages, whch can be used as gold standards for quanttatve evaluaton of retnal vessel segmentaton algorthms. The presented method was evaluated usng a new hgh-resoluton retnal mage database and also the wdely used DRIVE and STARE databases as representatve examples of the state-of-the-art. A quanttatve analyss shows that the results are at least comparable wth other recently publshed methods and even outperform most of them. Moreover, the presented method demonstrated the feasblty of relable detecton of the retnal vascular tree even n cases of glaucoma and dabetc retnopathy, whch affects the retna causng neovasculartes and hemorrhages. Ths latter capablty was revealed durng the evaluaton on the new hgh-resoluton database

23 Ths part s organzed as follows. Secton 2.2 gves an overvew of the state-of-theart n the feld of retnal vessel segmentaton algorthms. Secton 2.3 descrbes common fundus mage databases for evaluaton of retnal vessel segmentaton and ntroduces a new database of hgh-resoluton fundus mages. Secton 2.4 presents, n detal, the concepts of the proposed blood vessel segmentaton algorthm. Expermental results and dscusson are provded n Secton 2.5, whle the conclusons are gven n Secton State-of-the-art There s a consderable body of work on automatc retnal vessel segmentaton. A survey can be found e.g. n [44]. Dependng on the underlyng approach, most of the proposed methods can be grouped nto one of the followng categores: trackng-based, machnelearnng-based, model-based, or flter-based. Trackng-based methods usually map out a vessel centerlne and globally trace the vascular tree from seed ponts accordng to a relevant crteron. For example, Vlachos and Dermatas [170] proposed a blood vessel segmentaton technque based on mult-scale lne-trackng. Ther trackng procedure starts from a small group of pxels derved from a brghtness-selecton rule. It proceeds by followng a condton defned by crosssectonal vessel profles. The method worked very well on healthy subjects, gvng comparable results wth state-of-the-art methods. However, ths approach partally faled for pathologcal eyes. Other representatve methods of trackng-based vessel segmentaton algorthms can be found n [163] and [49], where a mult-drectonal graph search approach and Hough transform were utlzed, respectvely. Machne-learnng methods usually nvolve the supervsed classfcaton of mage pxels as ether belongng to retnal vessels or not. Such classfers are traned on a dataset of mage pxels, whch have already been hand-labeled as retnal vessels or non-vessels. Staal et al. [156] proposed a rdge-based blood vessel segmentaton method n combnaton wth a supervsed classfcaton technque. The method extracts mage rdges accordng to the vessel centerlnes, assumng that the vessels are elongated structures. Image rdges, thus, form a set of prmtves, whch are further classfed by a supervsed classfer. The proposed classfer was traned and evaluated on 40 mages of the DRIVE database. Overall, the method relably segmented the blood vessels. However, t also generated a number of falsely detected objects, probably due to the lmted set of tranng

24 data. The author noted that more tranng data mght lead to an mprovement n accuracy. Another classfcaton-based algorthm, whch was also traned on the DRIVE database, was proposed by Rcc and Perfett [143]. They proposed the use of a support vector machne (SVM) for classfyng the vessel and non-vessel pxels. Marín et al. [100] developed a supervsed classfcaton-based technque, whch used a 7-dmensonal feature vector composed of gray-level and nvarant moment-based features. They then classfed vessel pxels usng an artfcal neural network. Ther method outperformed most of the other approaches, especally n the case of pathologcal retnas. Another recent approach that employed the AdaBoost classfer was proposed by Lupascu et al. [96]. A 41-dmensonal feature vector was used for encodng nformaton on the local ntensty structure, as well as the spatal propertes and geometry of blood vessels at multple scales of the mage. In general, although recent supervsed-learnng based methods provde overall good results, they typcally utlze (for tranng) low-resoluton mages, resultng n low senstvty to thn retnal vessel detecton. A varety of methods falls nto the category of model-based retnal vessel segmentaton. Consder, for example, Jang et al. [76] who proposed an adaptve local thresholdng method, whch utlzed a verfcaton-based mult-threshold probng scheme. They segmented vessel structures usng hypothetc threshold values, whch were then evaluated by a general verfcaton procedure desgned accordng to the specfc propertes of the object of nterest; n ths case, retnal vessels. A method proposed by Al- Dr et al. [5] used a Rbbon of Twns actve contour model for segmentng and measurng blood vessel structures n fundus mages. Ths model employed two pars of contours to capture partcular blood vessel edges. The contours, themselves, were ntalzed va a generalzed morphologcal flter, whch dentfed the vessel centerlnes. Recently, Delbass et al. [29] publshed a new method for the segmentaton of vascular trees and the calculaton of blood vessel dameter and orentaton. The algorthm les on the borderlne between model-based and trackng-based technques. A generc parametrc model of retnal vessels was utlzed. Then an automatc blood vessel trackng algorthm was appled for tracng the vessels and determnng the vessel dameter. The method was adjusted usng a tranng subset of the DRIVE database for maxmzng vessel segmentaton accuracy and was evaluated usng a test subset of DRIVE. The method was compared wth sx other methods and outperformed three of them n terms of segmentaton accuracy. Lam et al. [88] publshed a retnal vessel segmentaton algorthm

25 desgned especally for segmentaton of mages of pathologcal retnas based on the dvergence of vector felds. In ths method, blood vessel-lke objects were extracted usng the Laplacan operator and nosy objects were pruned accordng to the centerlnes, whch were detected usng the normalzed gradent vector feld. An addtonal method proposed by Lam et al. [87] was specfcally developed for the segmentaton of retnal mages contanng brght and dark lesons. It utlzed a regularzaton-based multconcavty model of retnal vessel segments excludng lesons wth a characterstc cross-sectonal profle. Zhu et al. [186] presented a unversal model-based scheme for modelng of the blood vessel ntensty profles wth varyng boundary sharpness. The algorthm was based on measurements of symmetry and asymmetry n the Fourer doman va a phase congruency approach. In flter-based methods, retnal vessel segmentaton s typcally performed usng mathematcal morphology operatons [43]. In ths way, pror-knowledge of the vessel shapes s utlzed to create a morphologcal structurng element, whch s used for flterng objects from background. Zana and Klen [183] publshed an algorthm that combnes morphologcal flters and specfc blood vessel shape nformaton to segment the blood vessels. They used mathematcal morphology to hghlght vessels wth respect to ther morphologcal propertes, such as lnearty, connectvty, wdth and a specfc Gaussanlke profle. Mendonça and Camplho [104] ntegrated four drectonal dfferental flters for vessel centerlne extracton and employed morphologcal operators for fllng blood vessel segments. Fraz et al. [42] proposed a combned approach usng morphologcal operators for the blood vessel skeleton detecton and multdrectonal morphologcal bt plane slcng for extracton of the whole vasculature. Palomera-Pérez et al. [132] developed a fast parallel mplementaton of a retnal vessel segmentaton algorthm based on multscale feature extracton and regon growng. The method proposed by Soares et al. [152] utlzed wavelet flters. Ther algorthm appled Gabor wavelets for blood vessel detecton at several scales. A Bayesan classfer was then used for dfferentatng vessel and non-vessel pxels. Furthermore, the well-known concept of matched flterng (MF) s also commonly utlzed n flter-based methods. The use of MF n retnal vessel segmentaton algorthms was frst ntroduced by Chaudhur et al. [69]. Snce then, a number of MF approaches have been proposed for retnal vessel segmentaton. Pror knowledge, that s assumng a Gaussan-shape blood vessel profle, s often employed. In ths case, several two-dmensonal Gaussan-shape masks n dfferent orentatons are

26 usually convolved wth the green channel of colour fundus mages. The flter responses are then thresholded and fused to generate a bnary map of the vascular tree. Ths concept has been recently publshed n [6], [7], [23], [63], [184], and [185]. Hoover et al. [63] mproved the tradtonal MF approach by usng regon-based propertes when decdng whether the matched regon s a vessel or not. Al-Raw and Karajeh [6] mproved the basc MF technque by combnng t wth genetc algorthms. Another approach proposed by Cnsdkc et al. [23] was nspred by real ant colones. They presented a novel hybrd model consstng of MF and an ant colony classfcaton algorthm. The method s desgned to overcome the defcences of usng MF alone, especally ts nsuffcency to recover all retnal vessels (partcularly capllares) n the mage. Nevertheless, the algorthm s tested usng the DRIVE database, whch contans low-resoluton mages, where capllares are barely vsble. A modfed scheme of MF has been recently presented by L. Zhang et al. [185]. They appled a local blood vessel cross-sectonal analyss usng double-sded thresholdng to analyze the local structures of fltered outputs. They, thus, reduce the false detecton rate of the blood vessels due to non-lnear edges. An addtonal method proposed by B. Zhang et al. [184] presents a standard MF scheme supplemented by the frst-order dervatve of Gaussan (FDOG). The vessels are detected by thresholdng the mage response to the MF, whle the threshold s adjusted accordng to the response to the FDOG. The method s presented as a relable approach, whch reduces the false postve detectons produced by the orgnal MF method. It s clamed, that the method also offers detecton of very fne vessel structures; however, the evaluaton s performed only on low-resoluton mages. Although there are many methods utlzng dverse approaches for segmentng the blood vessel tree on fundus mages, they are developed and also ultmately tested usng publcly avalable, but low-resoluton mage databases (e.g. DRIVE or STARE). The true capacty of these methods to segment very fne vascular structures, ncludng capllares and neovascularsed blood vessels, remans an unknown and s often lmted. Applyng them on mage data of hgher resoluton may be only a queston of parameter adjustment for some methods. However, not all methods can be easly adapted to hgher resoluton data. A number of them wll probably fal on current hgh-resoluton fundus mages. Fundus mages usually exhbt specular reflectance on thck blood vessels. In ths case, flter-based methods may fal because of the wrong nterpretaton of specular reflecton as a blood vessel edge. Some authors try to solve ths problem by

27 downsamplng the orgnal mage, for example as n [16], [186]. However, ths mght lead to loss of nformaton n retnal vessel structures or n ndcators for retnal pathologes. Wth respect to the current state-of-the-art, ths dssertaton ams to contrbute wth a methodology for precse detecton of the retnal vessel structures n a wde range of vessel wdths n currently avalable hgh-resoluton fundus mages. The method allows detecton of fne blood vessel structures wth dameter ~ 5 pxels. The proposed technque also solves a common problem of the flter-based methods wth the specular reflectance on the large (thck) retnal vessels, and s able to segment the largest retnal vessels as a sold structure wthout artfacts due to the ncorrect matchng of the edges. Next, a new hgh-resoluton fundus mage database of healthy and pathologcal eyes s ntroduced n the frame of ths dssertaton as a further contrbuton to the current stateof-the-art n retnal vessel segmentaton. Gold standard mages of manually segmented blood vessels are also provded as part of the database. Thus, the database s ntended for utlzaton not only n the development and performance evaluaton of the proposed method, but also as a benchmark dataset for other authors gvng them a possblty to evaluate and compare ther vessel segmentaton algorthms. 2.3 Fundus mage databases for testng of the blood vessel segmentaton algorthms Three databases of retnal mages are utlzed for evaluaton of the blood vessel segmentaton n ths dssertaton. Prmarly, the new HRF (Hgh-Resoluton Fundus) mage database has been created as a new possblty for evaluaton of the method s performance. However, the most commonly used and probably the most known DRIVE (Dgtal Retnal Images for Vessel Extracton) [156] and STARE (STructured Analyss of the REtna) [63] databases are utlzed also for evaluaton of the proposed method and partcularly for the comparson wth other recent approaches n the lterature DRIVE The DRIVE database [156] has been establshed to enable comparatve studes on segmentaton of the blood vessels n retnal mages. The mage data for DRIVE database were obtaned from a dabetc retnopathy screenng program n The Netherlands. The

28 database contans 40 mages dvded nto tran and test set; both contanng 20 mages. 33 mages of the whole database do not show any pathologcal changes and 7 mages show sgns of mld early dabetc retnopathy. The mages were acqured usng CANON CR5 non-mydratc 3CCD camera wth 45-degree feld of vew (FOV). Each mage has sze of pxels and s stored n 24-bts colour space wthout any mage compresson. The database contans sngle manual segmentaton of the blood vessels for each mage n a tran set and two manual segmentatons for each mage n a test group. Bnary mask determnng FOV s provded for each mage. Example of the mage from the DRIVE database wth correspondng gold standard segmentatons s shown n Fgure 2. a) b) c) Fgure 2. A fundus mage "01_test.bmp" from a tran set n the DRIVE database: a) an orgnal RGB mage, b) correspondng manual segmentaton of the 1 st and c) 2 nd human observer STARE The STARE database has [63] been created at the Unversty of Calforna, San Dego, USA to enable comparatve studes of dfferent methods amed to support dagnoss of dfferent retnal pathologes. The database contans 20 selected fundus mages wth correspondng hand-labeled blood vessel segmentatons. The analog data were captured by a TopCon TRV-50 fundus camera wth 35-degree FOV. The data were dgtzed to produce mages wth pxels and stored n 24-bts colour space wthout any mage compresson. 10 mages are of subjects wth no retnal pathology. 10 mages contan pathology that obscures or confuses the blood vessel appearance n varyng portons of the mage. Each of these 20 mages was hand-labeled to produce a ground truth vessels segmentaton. The mages were labeled by two dfferent observers so two manual segmentatons are avalable (Fgure 3)

29 a) b) c) Fgure 3. A fundus mage "m0001.ppm" from the STARE database: a) an orgnal RGB mage, b) correspondng manual segmentaton of the 1 st and c) 2 nd human observer HRF As a part of ths dssertaton, the HRF database has been newly establshed durng the cooperaton wth Pattern Recognton Lab at the Unversty of Erlangen-Nuremberg, Germany 1 and Tomas Kubena's Ophthalmology Clnc, Zln, Czech Republc, where mages were acqured. The goal of ths dataset s to support comparatve studes on automatc segmentaton algorthms on retnal mages, especally hgh-resoluton ones. The database s onlne and can be downloaded from publc webstes 2. The database contans three sets of fundus mages: of healthy, glaucomatous, and dabetc retnopathy subjects. The frst set conssts of 15 mages of healthy subjects wthout any retnal pathology. The second set ncludes 15 retnal mages of patents wth dabetc retnopathy (DR) contanng pathologcal changes, such as neovascular nets, hemorrhages, brght lesons, spots after laser treatment, etc. The last group conssts of 15 mages of patents wth glaucoma n advanced stage wth symptoms of focal and dffuse RNFL loss. The second and thrd groups, thus, allow evaluaton of the segmentaton methods n the case of pathologcal retnas. All fundus mages were acqured wth a mydratc fundus camera CANON CF-60 UV equpped wth CANON EOS-20D dgtal camera wth a 60-degree FOV. The mage sze s pxels. Standard mydratc drops were used to dlate the subjects' pupls. All mages are 24-bts per pxel (true color) and are stored n JPEG format wth low compresson rates, as s common n ophthalmologcal practce. For each mage, 1 The cooperaton wth foregn partner was supported by blateral Czech-German grants no. D10-CZ16/09-10 and no. 7AMB12DE002 n the years and , respectvely

30 a bnary mask determnng the FOV s provded, snce the analyss s usually performed only n the nner area of the mage, surrounded by dark background (Fgure 4). The mages were segmented manually and ndependently by the author and partally also by other experts workng n the feld of retnal mage processng. They were traned by experenced ophthalmologsts from collaboratng clncs and they were asked to label all pxels belongng only to retnal vessels. ADOBE Photoshop CS4 mage edtor was used for manual labelng of the mages (Fgure 4). a) b) c) Fgure 4. Examples of fundus mages from the HRF database: a) mage "06_h.jpg" from the healthy group, b) mage "04_dr.jpg" from the DR group, and c) mage "14_g.jpg" from the group of glaucomatous mages wth correspondng hand-labeled gold standard segmentatons

31 2.4 Methodology An overvew of the proposed methodology to blood vessel segmentaton s as follows (see flowchart n Fgure 5). The preprocessng step conssts of llumnaton correcton and contrast equalzaton of the fundus mages n preparaton for further analyss. Only the green channel of an RGB mage s utlzed, snce ths channel has the hghest contrast between the blood vessels and other retnal structures (Fgure 6). The segmentaton of blood vessels n the preprocessed mage utlzes a MF approach. Fve two-dmensonal flters were desgned accordng to typcal blood vessel cross-sectonal ntensty profles; whereas fve dfferent blood vessel wdths were consdered from the thnnest to the thckest retnal vessels. The preprocessed mage s convolved wth each of the 5 kernels, each of whch s rotated nto 12 dfferent orentatons. The resultng parametrc mages are then fused so that the locally maxmum response s selected for each pxel. The fused parametrc mage s then thresholded n order to obtan a bnary map of the blood vessel tree. Ths s further cleaned to remove small or short artfacts due to nose or other mage structures that do not belong to the vascular tree. I(,j) * h 1,0 (x,y) MFR 1,0 (,j) G(,j) Illumnaton correcton and contrast equalzaton I(,j)... max{mfr k, (,j)} O(,j) Thresholdng O T (,j) Artefacts cleanng O F (,j) I(,j) * h k, (x,y) MFR k, (,j) Fgure 5. Flowchart of the proposed blood vessel segmentaton approach; where G(,j) green channel of the nput RGB mage, I(,j) preprocessed mage, h k,(x,y) convoluton masks (k=0,1,,4 and =0,15,,165 ), MFR k,(,j) matched flter responses, O(,j) resultng (fused) parametrc mage, O T(,j) thresholded parametrc mage, O F(,j) fnally cleaned mage. Fgure 6. Partcular colour channels of the mage 06_h.jpg from the HRF database

32 2.4.1 Illumnaton correcton and contrast equalzaton A B-splne based llumnaton correcton method s used as a preprocessng step to mprove the accuracy of the proposed segmentaton method. A non-unform llumnaton correcton s appled together wth contrast enhancement. Let s denote a green channel of the orgnal RGB fundus mage (Fgure 7a) as a greyscale mage G(,j) of sze M N, where =1,,M and j=1,,n are spatal coordnate ndces. A multplcatve llumnaton model s utlzed and the corrected mage I(,j) (Fgure 7b) s then gven as [113]: G (, j) I (, j) b max 128, (2.1) B (, j) where B(,j) s the background llumnaton model and b max s the maxmum ntensty value n the mage. The term ( b max +128) ensures that the mean value of the reconstructed mage wll be approxmately 128 (for mages wth 256 gray levels). The background llumnaton model s obtaned by approxmatng the low-pass fltered mage (derved from G(,j)) by a two-dmensonal B-splne functon as n [31] (Fgure 7c). The kernel for the low-pass flterng s a smple averagng flter wth pxels for a gven mage sze. The sze of the kernel was chosen heurstcally to suppress suffcently hghfrequency components n the mage (e.g. blood vessels, exudates, hemorrhages, etc.). a) b) c) Fgure 7. Preprocessng of the nput mage: a) green channel of the orgnal mage 05_dr.jpg from the HRF database, b) correspondng corrected mage, and c) two-dmensonal B-splne functon Two-dmensonal matched flterng The two-dmensonal matched flterng locally explots the correlaton between local mage areas and 2D masks developed accordng to the appearance of typcal blood vessel segments of dfferent wdths (dameters) and orentatons. These masks were created by measurng numerous perpendcular cross-sectonal ntensty profles of retnal vessels n the mages from the HRF database. The cross-sectonal profles were heurstcally

33 classfed nto fve classes (ndexed as k=0,1,,4) of dfferng blood vessel thcknesses to acheve a relable and precse detecton of all possble blood vessel segments wth an acceptable wdth resoluton. Thus, 250 profles were manually selected per blood vessel wdth class from all mages n the database, whereas maxmally 6 profles were selected from each mage. For each wdth class, all ts cross-sectonal profles were centered and then averaged n order to obtan a smoothed ntensty profle for that class. The resultng averaged profles cover a range of blood vessel dameters from 5 to 22 pxels, measured at a full-wdth at half-maxmum of the cross-sectonal profles (Fgure 8). The shape of the partcular ntensty profles smeared by plan parallel back projecton along the vessel axs thus represents peces of retnal blood vessel structures from the thnnest blood vessels (Fgure 8a) through thcker structures (Fgure 8b, Fgure 8c) to the thckest ones, whch appear wth central lght reflecton (Fgure 8d, Fgure 8e). The correspondng mask szes are 14 14, 22 22, 24 24, 26 26, and pxels. Hence, the partcular wdth classes cover all types of blood vessels n a common fundus mage. Fgure 8. Depcton of averaged blood vessel cross-sectonal profles (at the top) expanded nto the correspondng 2D kernels (at the bottom) classfed nto fve classes of dfferent wdths: from the narrowest a) to the wdest e) blood vessel profle; ndces m, n stay for spatal coordnates of partcular matrces. Partcular kernels were then rotated nto the angular drecton =0,15,,165 n order to cover suffcently all possble orentatons of the blood vessel segments. A square shape of the obtaned masks s assumed. Pxel values at ndvdual postons, whch do not ft the mage lattce durng mask rotaton, are blnearly nterpolated. Thus, a number of 12 dfferently orented kernels hk,(x,y) for each of the wdth classes s obtaned. The square shape of the 2D kernels was utlzed as a compromse between sgnal-to-nose rato (low for masks wth short length) and maxmal possble length of the blood vessel

34 segment fulfllng a condton of pecewse parallel edges. Furthermore, each kernel s convolved wth the preprocessed mage I(,j): MFR ( * y k, k,, j) I (, j) h ( x, ). (2.2) Thus, we obtan a number of 60 (5 12) parametrc mages (MFRk,(,j) matched flter responses) related to the correspondng wdth class k and the orentaton of blood vessel segment. The magntudes of the parametrc mages thus correspond to the degree of correlaton between partcular masks and local areas n the mage. The maxmum flter response ndcates the mask best matchng the wdth and orentaton of blood vessel segment contaned n the respectve mage area. Non-exstence of the blood vessel n the area s ndcated by a relatvely low value of the flter response. A jont parametrc mage O(,j) s then obtaned by selecton of maxmum response from the set of parametrc mages for partcular pxels: ( k, k, O, j) max MFR (, j). (2.3) The resultng parametrc mage s shown n Fgure 9a (for a fundus mage n Fgure 4a) Thresholdng and postprocessng The resultng parametrc mage s thresholded n order to obtan bnary representaton of the vascular tree. The blood vessels are consdered as a foreground (objects) and remanng parts as a background of the mage; whereas only pxels nsde the FOV are nvolved. The approach utlzed for thresholdng belongs to the class of mnmum error thresholdng methods [80], [147]. These methods assume that the mage can be formally characterzed by mxture denstes of foreground and background pxels [147]: p ( q ) P ( T ) p ( q ) [1 P ( T )] p ( q ). (2.4) f In ths equaton, p(q), q=0,...,q, where Q s the maxmum lumnance value n the mage, s referred to as a probablty mass functon (PMF). The terms pf(q), 0 q T, and pb(q), T+1 q Q, where T s the threshold value, are thus the PMFs of the foreground and background pxels, respectvely. P(T) s the cumulatve probablty functon defned as T P ( T ) p ( q ). The PMF p(q) can be smply estmated from one-dmensonal mage q 0 hstogram by normalzng t to the total number of samples [147]. b

35 The Kttler mnmum error thresholdng method [80], whch s presented n ths dssertaton, assumes that the pxel values of object (=f) and background (=b) are normally dstrbuted wth parameters mean µ and varance σ 2, derved accordng to the equatons [80]: where b 1 ( T ) q p ( q ), (2.5) P ( T ) b q a 2 2 ( T ) q ( T ) p ( q ), (2.6) P ( T ) 1 q a P ( T ) p ( q ), b q a a T 0 1 f b, (2.7) T f b. (2.8) Q b An optmal threshold can then be derved by an teratve computaton of the followng equaton [147]: T opt arg mn P ( T ) log ( T ) 1 P ( T ) log ( T ) f P ( T ) log P ( T ) 1 P ( T ) log 1 P ( T ), (2.9) b where σf and σb are foreground and background standard devatons, respectvely. Then, the thresholded mage OT(,j) s obtaned by thresholdng the values of O(,j) usng Topt. Other thresholdng methods were tested as well. Partcularly, the method based on evaluaton of local entropy from co-occurrence matrx [68] and the method utlzng standard Otsu approach [131] were tested. These methods were mplemented earler and some results were already publshed n [117], [119], and [120]. Then, the results of partcular thresholdng algorthms were evaluated and compared to each other usng the mages from the HRF database. Furthermore, t was found expermentally that the Kttler mnmum error thresholdng technque s the most relable for the proposed segmentaton task and hence ncluded n ths dssertaton

36 Fnally, morphologcal cleanng of the bnary mage OT(,j) s appled, by deletng the unconnected objects wth pxel area less than selected value (generally determned n heurstc manner to 200 pxels), to remove subtle artfacts that are not connected to the blood vessel tree. An example of the bnary representaton of blood vessels s shown n Fgure 9b and the correspondng morphologcally cleaned mage s n Fgure 9c. Fgure 9. Depcton of the blood vessel segmentaton results: a) resultng parametrc mage O(,j) composed from the partcular parametrc mages selectng the maxmum response for each pxel, b) thresholded mage O T(,j) obtaned by the proposed thresholdng algorthm, and c) morphologcally cleaned bnary mage O F(,j) (the mages correspond to the fundus mage n Fgure 4a). 2.5 Results and dscusson Evaluaton methodology The presented blood vessel segmentaton method was prmarly evaluated usng the new HRF database contanng hand-labeled mages. Secondly, evaluaton usng DRIVE and STARE databases was carred out as well. Inspred by other authors, the method performance was evaluated n terms of senstvty (SE), specfcty (SP), accuracy (ACC) and usng recever operatng characterstc (ROC) curve [38]. SE TP P TP TP FN TPF, (2.10) SP TN N TN TN FP, (2.11) ACC TP TP FN TN TN FP, (2.12) where TP (true postves) a number of pxels correctly detected as the blood vessel pxels,

37 FN (false negatves) a number of pxels ncorrectly detected as background pxels, TN (true negatves) a number of pxels correctly detected as background pxels, FP (false postves) a number of pxels ncorrectly detected as the blood vessel pxels, P a total number of the blood vessel pxels n a gold standard manual segmentaton, N a total number of background pxels n a gold standard segmentaton. The senstvty and specfcty measure ablty of the method to detect correctly the blood vessel and the background pxels, respectvely. Accuracy ACC can be characterzed as an overall measure provdng the rato of total well-detected pxels accordng to gold standard hand-labeled segmentaton. ROC curve s a plot of true postve fractons TPF versus false postve fractons FPF = 1 SP, as the grey level threshold of the thresholdng algorthm s vared [38]. From each ROC, an area under the curve (AUC) s computed. The closer the ROC curve s to the top left corner, the better s the performance of the method to match the blood vessels correctly by desgned flters. Thus, the deal value of AUC s equal to one Evaluaton of the method usng HRF database The proposed method was prmarly desgned for hgh-resoluton mages and therefore, n the frst nstance, t was evaluated usng the new HRF database. All sets of mages,.e. mages of healthy eyes, mages wth sgns of dabetc retnopathy (DR) and glaucoma were used. Table 1 Table 3 show results of the parameters SE, SP, ACC, and AUC evaluated on partcular datasets. Only pxels nsde the FOV were consdered n evaluaton of each mage. Mnmum and maxmum values are boldfaced for each parameter n these tables. Mean values and standard devatons are computed for each parameter as well. Consderng mean values at the bottom of the tables, t can be seen that the results for all datasets are comparable. Takng nto account a quanttatve comparson of average values n the sense of accuracy, the ACC values ndcate more than 94 % of correctly classfed pxels n the FOV for all datasets. A lower value of senstvtes n comparson to the other parameters s probably caused by lower ablty of the method to detect the fnest blood vessel structures (.e. thn capllares) wth cross-sectonal dameter approx. 2 3 pxels occurrng to some extent at the level of nosy background n the mage. The very thn blood vessel structures are usually hard to recognze vsually, even by an experenced human observer (see Fgure 14). Snce ACC can be regarded as an overall measure of the method performance, Fgure 10 shows sx examples of the mages of

38 healthy, DR and glaucomatous retnas (two for each set) representng mnmum and maxmum values of the ACC parameter n each set. It can be seen, that the segmentaton results of DR mages contan more artfacts. Ths s probably due to ncorrect matchng to pathologcal structures of these mages (see e.g. mage 06_dr.jpg n Fgure 10b temporarly from the OD), whch decreases specfcty of the method. Nevertheless, ths lmtaton of the method can be elmnated usng separate leson detecton approaches (for example as n [115], [8]) to exclude these pathologcal areas from the blood vessel segmentaton. Varyng the threshold at whch the resultng parametrc mage s thresholded, we can obtan the ROC curve and the correspondng parameter AUC by ntegratng the area under the curve. AUC was computed for each mage separately n each dataset; ther mean values ± standard devatons are shown n Table 1 Table 3. Resultng ROC curves for partcular mages and for each dataset are plotted n Fgure 11a c. Consderng the mnmum and maxmum values of AUC, t ndcates that the method performance s well comparable wth the state-of-the-art methods n the lterature (see Secton 2.5.4). Moreover, small values of the standard devaton of AUC ndcate rather good robustness of the proposed method; ths can be revealed also subjectvely by low varance of the areas under the plotted curves n Fgure 11. Next to a quanttatve evaluaton of the method performance, t was also examned qualtatvely, f the applcaton of fve dfferent kernels s reasonable wth regard to a suffcent wdth resoluton. Fgure 12 shows resultng blood vessel tree segmentaton from Fgure 9c. The blood vessel pxels are labeled accordng to the maxmal flter response belongng to the fve blood vessel wdth classes. It can be seen that the MF desgned for the thnnest blood vessels (ndcated by the red colour) has correctly maxmum response mostly n the center of the mage (macular regon). Thus, t agrees wth pror anatomcal assumptons. It can be also clearly observed that the maxmal responses follow up the vessel dameter n drecton from the macular regon to the wder blood vessel structures. Inspecton of the thckness map n Fgure 12 n detal (bottom part of the fgure) reveals an mportant behavor: the wde blood vessels wth a specular reflecton are ndcated by the correspondng blue colour mostly on the vessel's central axs, whch s often surrounded by the red lnes ndcatng narrow structures. It s qute natural as the maxmum response of the wde flter appears only for a good match near to the blood vessel central axs, whle the margnal strpes may appear as narrow blood

39 vessels; compare the narrow profle n Fgure 8a wth any of the symmetrcal halves of the thck profle n Fgure 8d or Fgure 8e. It s an advantage of the proposed method when utlzng dfferent matched flters wth dfferent shapes, thus enablng segmentaton of the blood vessels wth and wthout specular reflecton n the vessel centerlne. Table 1. Performance evaluaton on the healthy group (HRF); mnmum and maxmum values of each parameter are boldfaced. Image no. SE SP ACC AUC mean std Table 2. Performance evaluaton on the DR group (HRF); mnmum and maxmum values of each parameter are boldfaced. Image no. SE SP ACC AUC mean std

40 Table 3. Performance evaluaton on the glaucomatous group (HRF); mnmum and maxmum values of each parameter are boldfaced. Image no. SE SP ACC AUC mean std a) b) c) Fgure 10. Resultng blood vessel segmentatons (top par of each subfgure) for a) healthy, b) DR, and c) glaucomatous group wth correspondng gold standard hand-labeled segmentatons (bottom par of each subfgure). The left- and rght-hand sde of each subfgure represents mages that acheved maxmum and mnmum value of ACC wthn the partcular group, respectvely

41 Fgure 11. ROC curves plotted for each mage separately concernng mages of a) healthy, b) DR, c) glaucomatous retnas from the HRF database, d) mages from the DRIVE database, and e) mages from the STARE database; ROC charts are zoomed to the top left corner for better vsual dfferentaton between partcular curves. Fgure 12. Colour labelng of the blood vessel pxels accordng to detected wdth class n the colour spectral scale: startng from the red for the narrowest class contnung to blue for the wdest class; top part: an overall colour labelng of blood vessel tree segmentaton result, bottom part: detals of colour labelng of the blood vessel pxels accordng to vessel dameter

42 2.5.3 Evaluaton of the method usng DRIVE and STARE databases DRIVE and STARE databases were ncluded nto evaluaton n order to compare the proposed method wth state-of-the-art methods, snce they have not been evaluated usng the new HRF database yet. All these methods consdered n the performance evaluaton have been mentoned earler n Secton 2.2. Due to the low-resoluton mages n DRIVE and STARE databases, t s unsutable to test all desgned fltraton masks for fve wdth classes. Hence, t was determned expermentally, that applyng only two of them s adequate to get acceptable results. The kernels were obtaned heurstcally from the profles for "wdth 1" and "wdth 2" (Fgure 8a, Fgure 8b) by downsamplng correspondng masks wth factor 2. Evaluaton on DRIVE and STARE was performed usng the test sets of these databases, each contanng 20 mages wth gold standard segmentatons provded by the frst human observer. Table 4 shows senstvty, specfcty and accuracy for each mage n DRIVE and STARE databases. Mnmum and maxmum values for each parameter are boldfaced n ths table. Fgure 13 then shows comparson of segmentaton results wth the mnmum and maxmum value of ACC together wth correspondng gold standard segmentatons for the DRIVE database. The test set of the DRIVE database contans two manual segmentatons provded by two human observers for each of the twenty mages. One of these manual segmentatons s usually used as a gold standard and the other one for comparson of automated segmentaton methods wth the human observer approach. The proposed method acheved good performance n comparson wth the human observer (Table 4 Table 5). The ACC parameter of the proposed method s slghtly lower than ACC of the human observer segmentaton. Nevertheless, the dfference between them s only , whch seems to be neglectng. Then, the dfference between average values of senstvtes of the proposed method and the human observer approach (0.0747) s apparently hgher than n case of accuraces. Ths s probably due to that challengng task to detect small blood vessels, snce they are hard to dstngush from the background nose. Further vsual nspecton revealed, that these structures are also barely recognzable by experenced human observer n the DRIVE database, see Fgure 14. Then, comparson of the method's senstvty to detect the thnnest blood vessel structures can be pretty confusng, even when the second human observer dffers from the frst one (Table 5)

43 As for the HRF database, the ROC curves are plotted also for the partcular mages of DRIVE (Fgure 11d) and STARE databases (Fgure 11e). AUC s computed for each mage and averaged along the number of 20 mages n the datasets (the last two columns n Table 4). Table 4. Performance evaluaton on DRIVE and STARE databases; mnmum and maxmum values of each parameter are boldfaced. Image SE SP ACC AUC no. DRIVE STARE DRIVE STARE DRIVE STARE DRIVE STARE mean std Fgure 13. Comparson of the best (left) and the worst (rght) results measured by ACC usng the DRIVE database; at the top: results of the proposed method, at the bottom: gold standard segmentatons

44 Table 5. Comparson of the 2 nd and the 1 st human observer segmentatons on the DRIVE database; mnmum and maxmum values of each parameter are boldfaced. Image no. SE SP ACC mean std st human observer s consdered as a gold standard segmentaton. Fgure 14. a) Secton of the green component of the mage 01_test.tf n the test set of the DRIVE database wth correspondng gold standard mages labeled by b) 1 st observer and c) 2 nd observer as a proof of dffculty of human observer to decde whether there s a vessel or not Comparson wth other methods Table 6 compares performance of the proposed method (note that t s a smplfed verson as descrbed above) wth 12 dfferent methods from the lterature, ncludng the frst MF approach [69]. The human observer approach s consdered as well. The gaps n ths table ndcate that the correspondng value was not provded by the author. The approaches are ordered accordng to the ACC parameter for the DRIVE database, snce ACC values are avalable for all of these methods. Sgn "MF" denotes that the method

45 utlzes matched flterng approach. As Table 6 shows, the proposed method (even n ts smplfed verson) s comparable wth other recent methods n the lterature. Nevertheless, t should be noted, that the proposed method s prmarly desgned and tested for hgh-resoluton fundus mages. Then, Table 7 shows results for the new HRF database. It can be observed, that the proposed method acheved much better performance usng all desgned flters together. Ths can be also compared wth the results n Table 6. Table 6. Comparson of the proposed method wth other vessel segmentaton algorthms evaluaton on DRIVE and STARE databases; mnmum and maxmum values of each parameter are boldfaced. Segmentaton method SE SP ACC AUC DRIVE STARE DRIVE STARE DRIVE STARE DRIVE STARE Lupascu et al., 2010 [96] Al-Raw et al., 2007 [7] (MF) nd human observer Soares et al., 2006 [152] Marín et al., 2011 [100] Staal et al., 2004 [156] Lam et al., 2010 [87] Zhang et al., 2010 [184] (MF) Delbass et al., 2010 [29] Proposed method (MF, smplfed verson) Cnsdkc et al., 2009 [23] (MF) Al-Dr et al., 2009 [5] Palomera-Pérez et al., 2010 [132] Chaudhur et al., 1989 [69] (MF) MF ndcates the methods based on matched flterng. Partcular approaches are ordered accordng to ACC for the DRIVE database. Table 7. Evaluaton of the proposed method usng the HRF database n comparson wth DRIVE and STARE databases; mnmum and maxmum values of each parameter are boldfaced. SE SP ACC AUC Proposed method (HRF healthy mages) Proposed method (HRF glaucomatous mages) Proposed method (HRF DR mages) Proposed method (DRIVE) Proposed method (STARE) Notes about method mplementaton The method was mplemented usng MATLAB (R2009b) programmng software. A set of expermental program modules was wrtten. Evaluaton of the algorthm was performed on a personal computer wth Intel Core 7 processor, 4 GB system memory,

46 and Wndows 7 Professonal 64-bt operatng system. An average computatonal tme of one mage for the HRF database was 92 seconds, 3.22 seconds for the DRIVE database, and 4.07 seconds for the STARE database. Longer tme needed for computaton of mages from the HRF database s caused by both the much hgher resoluton of mages n ths database and utlzaton of all desgned kernels. However, t must be noted that mplementaton of the presented method has not been optmzed concernng computatonal complexty. To acheve better computatonal performance, dfferent programmng languages and parallel mage processng technques should be consdered for mplementaton of the proposed method. One of the man advantages of ths method s a good possblty to mplement the algorthm usng parallel computng technques, snce only multple convoluton operatons are needed. 2.7 Concluson A method for effcent and relable retnal vessel segmentaton usng colour fundus mages s ncluded n ths dssertaton. The proposed approach utlzes matched flterng and mnmum error thresholdng technque to extract bnary blood vessel tree. Fve dfferent kernels were desgned accordng to typcal blood vessel cross-sectonal profles consderng fve wdth classes of retnal vessels to cover all blood vessel structures n commonly utlzed fundus mages wth relable wdth resoluton. One of the man prortes of the method s also capablty to segment the blood vessels wth specular reflecton. The method was quanttatvely evaluated usng the publcly avalable DRIVE and STARE databases and t has been showed that the results are comparable wth other recent methods n the lterature, even f a smplfed verson of the proposed method has been consdered. Besdes that, the new retnal database HRF of hgh-resoluton fundus mages of healthy subjects and subjects affected by dabetc retnopathy and glaucoma has been ntroduced. Correspondng gold standard mages were created for each fundus mage n the database by manual labelng of the blood vessel tree. The database s freely avalable onlne. Thus, t provdes a novel opportunty to researchers workng n the feld of retnal mage processng to evaluate ther blood vessel segmentaton methods. Today, the HRF database conssts of three sets of mages (mages of healthy, DR and glaucomatous retnas). It s ntended to add further gold standard data to the exstng mages to help the

47 evaluaton of blood vessel segmentaton algorthms amed to dfferentate between arteres and vens and measurng vessel dameters as well. Also t s ntended to expand ths database, not only for evaluaton of retnal vessel segmentaton algorthms, but as well as to help researchers to evaluate methods focused on other tasks n the area of retnal mage processng, for example glaucoma dagnoss, segmentaton of the optc dsc or the fovea, and localzaton of vessel bfurcatons and vessel crossngs

48 PART II 3 ANALYSIS OF FUNDUS IMAGES FOR RETINAL NERVE FIBER LAYER ASSESSMENT 3.1 Background As mentoned prevously n Secton 1, glaucoma s regarded as one of the most common causes of permanent blndness worldwde. Pathologcal changes due to ths dsease are permanent and unfortunately cannot be reversed. It s therefore extremely desrable to detect the dsease early to be able to start an approprate treatment as soon as possble. The glaucoma s characterzed by degeneraton of the retnal nerve fbers. Ths s usually also accompaned by an ncreased ntraocular pressure [39]. Loss of the nerve fbers results n decrease of the retnal nerve fber layer (RNFL) thckness. Then, the connecton between the photoreceptors and the bran s progressvely reduced and the patent loses hs vson. Pathologcal changes n the RNFL affects also structural appearance of the optc nerve head (ONH) the neuroretnal rm becomes thnner and the cup expands due to loss of the nerve fbers (Fgure 15). The qualtatve evaluaton of the ONH morphology and the RNFL structure together wth varous permetrc tests and measurements of ntraocular pressure are the common parameters that are used for glaucoma dagnoss n medcal practce nowadays [92]. However, these subjectve tests and vsual evaluatons lead to large nter- and ntra-observer varablty when dfferentatng between the normal and glaucomatous retnas [168]. Therefore, a quanttatve computer-based analyss of fundus mages can contrbute to make general qualtatve assessment more objectve and reproducble. Quanttatve parameters of the ONH morphology can be obtaned usng dfferent magng modaltes. For example, mportant quanttatve characterstcs of the ONH (dsc area, dsc dameter, rm area, cup area, or cup dameter) can be derved from stereo fundus photographs [12]. Then, the well-establshed and well-known cup/dsc rato (C/D) can be

49 computed from these morphologcal characterstcs too [10]. Ths rato compares dameter of the cup wth the total dameter of the optc dsc,.e. the ONH. Thus, for the glaucoma assessment, C/D can quantfy thnnng of the neuroretnal rm (Fgure 15). In addton, other magng modaltes (as dscussed n Secton 1) can be used to establsh quanttatve parameters of the ONH for glaucoma dagnoss as well: CSLO (HRT), SLP (GDx), or OCT [50], [103], [148]. Wdely used HRT provdes 2.5-dmensonal topographc mages of the ONH. Ths devce allows to generate morphologcal parameters such as the cup volume, cup depth, cup shape, or retnal heght varatons along the rm contour, whch are processed further by so-called Moorefeld s regresson analyss to classfy between normal and glaucomatous cases [106], [159]. Glaucoma probablty score can be computed by the newest model HRT III from these ONH parameters furthermore [4], [18]. a) b) Fgure 15. Major structures of the ONH for a) healthy eye, b) glaucomatous eye. Snce glaucomatous changes occur frst n the retnal nerve fbers, assessment of the RNFL s much more senstve for early detecton of glaucoma than only an evaluaton of the ONH. Hence, current magng modaltes target on an assessment of the RNFL prmarly. Among many modaltes, OCT s now regarded as gold standard glaucoma dagnostc devce, snce t allows drect measurement of the RNFL thckness. Unfortunately, the OCT devce s stll qute expensve and not generally avalable for many ophthalmc facltes around the world. On the other hand, acquston procedure by fundus camera s much faster and cheaper. Therefore, fundus camera s stll regarded as a fundamental dagnostc devce that can be used for glaucoma dagnoss

50 The retnal nerve fber layer can be observed as a unform fnely strated radal pattern appearng on the background of red-free fundus photographs (Fgure 16a). Due to the RNFL atrophy that causes contnuous decrease of the RNFL thckness, the RNFL pattern begns to change consequently. Ths has been proven by several studes, although so far evaluated by subjectve methods only [66], [153]. Then, n the advanced state of glaucoma dsease, the pattern s completely mssng (Fgure 16b). These facts brng an dea to evaluate the RNFL pattern by mage processng methods that could enable detecton of pathologcal changes before any substantal damage occurs. In addton, the potental of fundus mages s stll ncreasng, snce spatal resoluton of current dgtal fundus cameras s gettng to be better. The RNFL pattern s qute well recognzable n current hgh-resoluton fundus mages and thus t offers utlzaton of advanced texture analyss technques to descrbe subtle changes n the RNFL. Unfortunately, an automatc computer-based approach for evaluaton of the progressve RNFL changes n fundus mages has not been so far developed. a) b) Fgure 16. Depcton of the RNFL strated pattern n the area around the ONH for a) healthy eye, b) glaucomatous eye wth dstnctve wedge-shaped RNFL loss. As a contrbuton to the recent state-of-the-art (Secton 3.2), ths dssertaton proposes a novel approach to the RNFL assessment va texture analyss n colour fundus mages. The approach utlzes Gaussan Markov random feld (GMRF) texture modelng and local bnary patterns (LBP) for descrpton of the RNFL texture. The method n dfferent states has been publshed at several conferences [116], [118], [124], [125], [126], and as a part of a journal paper [73]. The complete work n the latest state as s covered by ths dssertaton has been submtted recently for a journal publcaton [127] as

51 well. Earler developng states of the proposed methodology were focused manly on dfferentaton between two margnal cases of texture representng healthy and glaucomatous tssue wth the total RNFL loss. Although the earler works proved good usablty of the proposed texture analyss for descrpton of the RNFL pattern, the am of the method to dfferentate roughly between two margnal cases has been later found as less effectve and mproper for possble clncal applcaton. Therefore, n contrast to the prevous states, the current state of the proposed methodology s more robust and orented towards descrpton not only of two margnal cases but also of subtle changes n the RNFL pattern. Thus, the approach n the latest state utlzes GMRF and LBP methods to generate texture features useful for capturng contnues varatons n the RNFL thckness and s able to predct the RNFL thckness wth help of dfferent regresson models. Then, the man contrbuton to the recent state-of-the-art s ablty of the proposed methodology to follow the RNFL thckness utlzng standard hgh-resoluton colour fundus mages. Snce the OCT devce enables drect measurement of the RNFL thckness, t s used as gold standard for valdaton of the proposed approach. The rest of Part II s organzed as follows. Secton 3.2 gves state-of-the-art n the feld of glaucoma dagnoss va colour fundus photography. Then, Secton 3.3 shows expermental mage data used for the development and evaluaton of the proposed method. Secton 3.4 ntroduces the methodology and the background of the proposed texture analyss approach. Results and dscusson are presented n Secton 3.5 and concluson of ths part s gven n Secton State-of-the-art There are many approaches dealng wth the analyss of common fundus mages to support glaucoma dagnoss va dfferent mage processng methods. There s a recent tendency to process fundus mages usng automatc computer-based methods n a smlar way as ophthalmologsts can do t vsually by nvestgaton of dfferent dagnostcally mportant retnal structures. Two retnal structures the ONH and the RNFL are used for dagnoss of glaucoma n general. Therefore, the methods to evaluate changes n the ONH morphology and the RNFL textural appearance represent a man core of current publcatons that are focused on glaucoma dagnoss usng fundus photographs

52 The assessment of the ONH structural appearance s usually ncluded as a part of commercal magng modaltes used n ophthalmology. Furthermore, some authors try to mprove glaucoma dagnoss va computer-based analyss of the ONH morphology n common fundus mages as well. For example, Inoue et al. (2005) [72] publshed a method amed to support evaluaton of the ONH morphology usng C/D parameter computed from the segmented ONH. Song et al. (2005) [154] presented a methodology for glaucoma screenng based on three aspects: the ONH morphology assessment, measurement of ntra-ocular pressure, and vsual feld examnaton. Then, a fuzzy logc s used for classfcaton of derved features. However, the contrbuton presents only a basc dea and does not contan any detals about the proposed methods and any results. Xu et al. (2006) [180] use snake contour model for the ONH segmentaton and compute C/D rato further. Nayak et al. (2009) [111] utlze C/D parameter and two addtonal features a rato of a dstance between the ONH center and the ONH margn to dameter of the ONH, and a rato of the blood vessel area n nferor-superor sde to area of the blood vessels n the nasal-temporal sde (ISNT parameter). These features are valdated by classfcaton of normal and glaucomatous cases usng neural network classfer. 61 mages (24 of normal and 37 of glaucomatous subjects) were consdered for ths evaluaton. The results showed 100 % senstvty and 80 % specfcty. However, the method was tested usng only low-resoluton fundus mages ( pxels). Bock et al. (2010) [14] proposed a method utlzng morphologcal parameters of the ONH for dervaton of GRI glaucoma rsk ndex. The analyss was performed on rather large dataset: 575 mages (336 of normal and 239 of glaucomatous subjects) wth classfcaton accuracy of 80 %. Next to the ONH morphology assessment, examnaton of the RNFL plays equally or even more mportant role, snce the loss of ganglon cells s a prmary event n glaucoma. Hstorcally, an attempt to utlze fundus cameras for glaucoma detecton by evaluaton of the RNFL appearance has been frst ntroduced by Hoyt et al. (1973) [64]. The authors descrbed the RNFL pattern n fundus photographs and compared ts changes wth permetrc fndngs. They revealed that these funduscopc sgns of the pattern provde the earlest objectve evdence of the RNFL atrophy n the retna. Lundström et al. (1980) [95] nvestgated black-and-whte photographs amed at subjectve evaluaton of the RNFL textural appearance as well. The authors proved the change of the RNFL pattern ntensty n fundus mages connected to the progresson of glaucomatous damage

53 Other authors have followed subjectve evaluaton of fundus photographs afterwards. Araksnen et al. (1984) [3] presented a subjectve method amed at evaluaton of the RNFL status n perpapllary area (surroundng the ONH). They nvestgated the RNFL pattern vsually and scored glaucomatous damage n a numercal scale (0 RNFL wthout any damage, 1 suspcon of the RNFL loss, 2 partal RNFL loss, 3 extensve RNFL loss, and 4 entre RNFL loss). In ths study, 142 subjects were evaluated (84 normal subjects and 58 subjects wth glaucoma). Pel et al. (1989) [134] performed one of the frst sem-automatc analyss of the RNFL texture usng dgtzed black-and-whte fundus photographs. In addton, they analyzed ntensty nformaton about the RNFL presence and tred to detect darkenng caused by the RNFL atrophy. However, only 5 mages of normal subjects, 5 mages of glaucomatous subjects, and 5 mages of subjects suspected of glaucoma were ncluded n ths study. Yogesan et al. (1998) [181] performed prelmnary analyss of 10 (5 normals, 5 glaucomatous) dgtzed fundus photographs (wth sze of pxels) va texture analyss based on gray level run length matrces. The method showed promsng results for large focal wedge-shaped RNFL losses that were well outlned by the surroundng healthy nerve fber bundles. Nevertheless, dffuse RNFL losses that generally affect the entre retna wth a complex decrease of the RNFL thckness could not be detected due to the low mage resoluton. Moreover, a lmted set of mage data nfluenced sgnfcance of the results. Tuulonen et al. (2000) [164] performed mcrotexture analyss of 7 normal subjects, 9 subjects wth dstnctve glaucomatous damage, and 8 subjects wth hgher ntraocular pressure (suspcous of glaucoma). Dgtzed fundus photographs wth sze of pxels were analyzed. The hypothess that changes n the RNFL structure can be seen as changes n mcrotexture of dgtal mages was nvestgated. Although they acheved some postve results, the dfferences were not statstcally sgnfcant because of small sample sze. An ntensty nformaton about the RNFL texture were utlzed agan by Dardjat et al. (2004) [28]. The authors evaluated ntensty dfferences n the RNFL texture along the ntensty profles between two concentrc crcles placed n the center of the ONH. The contrbuton presents an approach for analyss, but number of subjects tested and any statstcs are not mentoned. Other contrbuton utlzng ntensty crteron was wrtten by Lee et al. (2004) [89]. The crcle s placed n the center of the ONH and the RNFL status s evaluated usng the ntensty profle measured on that crcle. Smlarly as n a prevous artcle, no statstcs s presented

54 Besde older artcles mentoned above, several other works concernng an assessment of the RNFL status n fundus photographs have been publshed recently. The other authors have been nvestgatng fundus photographs n more or less smlar way. In the case of glaucomatous damage, the RNFL appears darker n fundus mages. Therefore, many authors tred to nvolve only ntensty crtera for decson tasks to detect glaucomatous changes, even n the recent state n ths feld of appled research [60], [107], [129], [141]. A plot study to search the RNFL thnnng n colour fundus mages wth sze of pxels was presented by Olva et al. (2007) [129]. The artcle presents sem-automatc method to texture analyss based on evaluaton of the RNFL pattern ntensty along 24 concentrc crcles centered n the ONH. Only small datasets of 9 mages of normal and 9 mages of glaucomatous subjects were tested. The results revealed correlaton between the ntensty-related parameters extracted from fundus mages and the RNFL thckness measured by OCT. Hayash et al. (2007) [60] used an approach wth Gabor s flters to enhance certan regons wth the RNFL pattern and cluster these regons whether there s glaucoma defect or not. The paper presents prelmnary results that were followed up by the same group n [107] (Muramatsu et al., 2010). Here, n comparson wth a prelmnary approach, the authors extended the concept of analyss and performed evaluaton usng larger dataset. 162 colour fundus mages were analyzed (81 mages of normal and 81 mages of glaucomatous subjects). Unfortunately, the mages had sze of pxels that s stll relatvely low-resoluton. These mages were used also to create manually labeled gold standard mages wth the RNFL losses depcted by two professonal observers. The method acheved 91 % senstvty. Nevertheless, the method s sutable only for detecton of focal and even wder RNFL losses expressed by sgnfcant changes n ntensty. Hence, the method s probably not sutable for detecton of the thn focal or even the dffuse RNFL losses. However, n spte of these lacks, ths s the frst journal publcaton presentng a fully automatc method for the RNFL assessment that s evaluated on rather large dataset. Prageeth et al. (2011) [141] publshed a method for glaucoma detecton usng ntensty crteron as well. They analyzed even larger dataset conssted of 829 (300 normals and 529 glaucomatous) fundus mages wth sze of pxels. Although, the results seem to be promsng, utlzaton of ntensty crtera used alone s not a good soluton. Intensty changes n the RNFL pattern can be detected only f the RNFL atrophy s so dstnctve than the patent has rather large vson loss already. Moreover, mage ntensty can be nfluenced by many factors, e.g. non-homogenous llumnaton, reflecton of the retna, homogenety of lght

55 power used for acquston, etc. In addton, recently Acharya et al. (2011) [2] proposed a method to analyze the RNFL texture usng hgher order spectra, run-length and cooccurrence matrces. The method was tested on 60 mages (30 normals and 30 glaucomatous). However, the mage sze was agan small ( pxels). The authors used several supervsed classfcaton technques to classfy normal and glaucomatous mages. Although, the classfcaton accuraces were more than 80 %, the artcle does not explan thoroughly a process of features extracton and whch areas of the mage were analyzed. Moreover, the mages presented n the paper seem to be of rather bad qualty and the RNFL texture s not apparent. Nevertheless, run-length and co-occurrence matrces have been shown as qute relable descrptors of the RNFL texture earler n a prelmnary study proposed by Yogesan et al. (1998) [181] as well as by our group (Kolar et al., 2008 [85]) at the Department of Bomedcal Engneerng (DBME), Faculty of Electrcal Engneerng and Communcaton (FEEC), Brno Unversty of Technology (BUT). Our group at the DBME, FEEC, BUT has been focused on retnal mage processng snce Here, some papers presentng the work that s not covered by ths dssertaton are referred. Dfferent methods of texture analyss towards descrpton of textural propertes of healthy and glaucomatous RNFL tssue n colour fundus mages were tested. These methods nvolve a range of approaches and uses dfferent texture features: mage ntensty, basc statstcs, run-length and co-occurrence matrces, edge representaton of the RNFL pattern, Fourer spectral analyss, or features of fractal dmensons. Some of these prelmnary results have been publshed at several conferences [47], [48], [74], [82], [85] as well as journal publcatons [75], [84], [86]. Although, there s a consderable range of artcles focused on analyss of fundus mages amed at glaucoma dagnoss, a complex methodology for the RNFL assessment n colour fundus mages s stll mssng. Many publshed artcles present methods based on evaluaton of the RNFL ntensty. As dscussed above, utlzaton of ntensty as a feature for detecton of changes n the RNFL s less robust and unsutable due to many physcal as well as physologcal reasons. Moreover, testng of the publshed methods s based manly on low-resoluton mages. Thus, subtle varatons n the RNFL thckness cannot be easly handled, snce the RNFL texture s not detaled enough due to the lowresoluton. The RNFL pattern s much more detaled and easly observed n current hghresoluton fundus mages. Ths offers a potental applcaton of advanced texture analyss

56 technques takng nto account not only ntensty crtera, but also varous spatal characterstcs of adjacent pxels n the texture. Hence, consderng the recent state-ofthe-art, ths dssertaton ams to contrbute wth a new and complex approach to the RNFL texture analyss that s able to capture contnues varatons of the RNFL thckness n up to date hgh-resoluton colour fundus mages. 3.3 Expermental mage database The mage database has been created on the bass of cooperaton wth the Eye clnc at the Erlangen Unversty Hosptal, Germany. The database so far contans 19 mage sets of healthy subjects wthout any sgns of glaucoma dsease and 8 mage sets of glaucomatous subjects wth focal wedge-shaped RNFL loss. Only one eye of each subject was maged. Each mage set contans an mage acqured by a common non-mydratc dgtal fundus camera CANON CR-1 (EOS 40D) wth 60-degree feld of vew (FOV). The mages have sze of pxels, whch s a common resoluton for many current fundus cameras. Standard CANON raw data format (CR2) and low-compressed JPEG wth 24-bt colour space (RGB) were used for storage of the mages. An example of RGB fundus mage of the healthy rght eye s shown n Fgure 17. Fgure 17. An example of orgnal RGB fundus mage of the healthy rght eye and partcular colour channels. In standard fundus mage, the red (R) channel appears oversaturated, whle the green (G) and the blue (B) channel show the blood vessels and retnal nerve fber layer well contrasted

57 The database then contans three-dmensonal volume data and crcular scans, whch were acqured by a spectral doman OCT system (Spectrals HRA OCT, Hedelberg Engneerng) for each of the 27 subjects 3. Infrared reflecton mages (scannng laser ophthalmoscope SLO) and B-scan (cross-sectonal) mages were acqured smultaneously by OCT dual laser scannng system. From 61 to 121 B-scans per one eye were taken, whch corresponds to the spacng between B-scans approxmately from µm to 63.1 µm (n 30-degree FOV). Acquston of the OCT mage volume (Fgure 18a) was performed wthn the perpapllary area. Crcular scan pattern (Fgure 18b) s usually used for glaucoma dagnoss va OCT. A crcle wth dameter 3.4 mm s placed n the center of the ONH and one sngle B-scan s measured along ths crcle [11]. a) b) Fgure 18. An example of OCT volume and crcular scans. a) SLO mage (left) wth the volume scan pattern allocated by the green lnes and one B-scan (rght) measured at the poston depcted by the blue lne n SLO; b) SLO mage (left) wth the crcular scan pattern defned by the blue crcle and the B-scan (rght) measured along ths crcle n drecton gven by the arrow. 3 For the purpose of evaluaton of the proposed method, all subjects were maged va fundus camera and consequently by Spectrals OCT magng system at the Eye clnc at the Erlangen Unversty Hosptal, Erlangen, Germany. Raw data from Spectrals OCT system needed to be extracted by specal research operatng software. The number of subjects has been so far lmted, snce acquston of mages (especally of glaucomatous subjects) requred ncreased workload of clncans/specalsts and costs per travel

58 3.4 Methodology An llustratve and concse schematc dagram of the proposed RNFL assessment methodology s depcted n Fgure 19. The texture analyss s performed wthn the perpapllary area at the locatons wthout the blood vessels only. For the blood vessel segmentaton, the proposed matched flterng approach descrbed n Part I s utlzed. Varous regresson models are tested towards predcton of the RNFL thckness usng the proposed texture features. The regresson models are traned on small square mage regons (ROIs) selected from fundus mages n the database and known measurement of the RNFL thckness. Crcular profles are extracted from the predcted mages provded by regresson models. The profles obtaned are further evaluated wth respect to the real RNFL thckness measured va OCT. The followng sectons deal wth descrpton of partcular methods for processng of mages as well as evaluaton approaches. Fundus mages (tranng dataset) REGRESSION MODEL ROIs selecton Texture analyss Regresson model RNFL thckness OCT volume scans RNFL thckness maps B-scans Fundus mages (testng dataset) OCT DATA OCT crcular scans Texture analyss Crcular profle OCT-SLO mage Blood vessel maskng Predcted mage Thckness profle B-scan Evaluaton Blood vessel segmentaton Fgure 19. Schematc dagram of the proposed methodology for the RNFL assessment

59 3.4.1 Data preprocessng Preprocessng of fundus mages The fundus mage data of healthy and glaucomatous subjects were preprocessed n several steps to obtan mages sutable for followng texture analyss. Frst, standard uncompressed TIFF format was reconstructed from raw data usng DCRAW freeware software [26], whereas a lnear transfer functon was used for the reconstructon. The most nformaton of the RNFL appearance les between the green (G) and the blue (B) spectral part of the vsble lght (see Fgure 17) 4. Hence, an average of G and B channel (further GB mage) was computed for each fundus mage. Further, only the GB mages were analyzed. Next, non-unform llumnaton of fundus mages was corrected together wth an ncrease of mage contrast usng CLAHE (Contrast Lmted Adaptve Hstogram Equalzaton) technque [139]. The fnal preprocessng step also results n enhancement of the RNFL pattern 5 (see an example n Fgure 20). The normal perpapllary RNFL appears as a unform strated textural pattern wth dfferent coarseness dependng on the angular poston around the ONH. The RNFL s much better vsble n the perpapllary area, snce there are the nerve fbers anatomcally more concentrated. For the frst testng and for the tranng of regresson models, squareshaped mage regons of nterest (ROIs) wth sze of pxels were manually extracted from all fundus mages ncluded n the group of normal subjects. Extracton of ROIs was performed unformly n the perpapllary area to the maxmum dstance not exceedng 1.5 dameter of the ONH; whereas ndvdual ROIs were not allowed to overlap and only locatons wthout the blood vessels were consdered (Fgure 20). In ths way, a number of 354 ROIs was collected (see few examples n Fgure 20). Partcular ROIs represent the typcal RNFL pattern that changes ts appearance accordng to the poston n the perpapllary area. The RNFL thckness vares dependng on the poston on the retna even for healthy subjects conformably to the anatomcal assumptons. Selecton of ROIs at dfferent postons thus covers a suffcent range of the RNFL thcknesses (approx µm) that can be used for tranng of regresson models. 4 Ths has been also nvestgated and showed by earler studes already at the DBME, FEEC, BUT [82], [84]. 5 Dfferent preprocessng technques for the RNFL enhancement were tested wth respect to the output of overall evaluaton. The dfferences between the overall results were rather small, even when an approach wthout any enhancement technque was used. However fnally, CLAHE was utlzed provdng slghtly better results than the other approaches

60 Fgure 20. The ONH surroundngs of the preprocessed fundus mage of a normal subject and ROIs depcted by the red colour. Magnfed examples of eght ROIs relatng to the dfferent postons n fundus mage are shown on the fgure s sdes Preprocessng of OCT data The OCT volume data were preprocessed n order to get the RNFL thckness n the perpapllary area of each subject. The RNFL was segmented and the correspondng RNFL thckness map was created usng the research software package OCTSEG (Optcal Coherence Tomography Segmentaton and Evaluaton GUI) for the OCT data segmentaton 6 [102]. Segmentaton of the layer was done automatcally and then only subtle manual correctons were made for each B-scan usng ths software package. Segmentaton result of the RNFL n one B-scan and the complete reconstructed thckness map can be seen n Fgure 21. The RNFL thckness usually vares from the thnnest (the blue colour) to the thckest (the red colour) structures as can be seen from Fgure 21a. 6 The OCTSEG software was developed at the cooperated Unversty of Erlangen by Markus Mayer, also as a part of blateral Czech- German grants no. D10-CZ16/09-10 and no. 7AMB12DE002 n the years and , respectvely. The author of ths dssertaton partcpated n evaluaton of the software performance

61 a) b) Fgure 21. OCT volume data: a) the RNFL thckness map mapped on the SLO mage; the color bar on the top shows values of the RNFL thckness measured n m, b) one B-scan acqured at the poston marked by the black lne n a); the RNFL s segmented between the red and the green curves Fundus-OCT mage regstraton A landmark-based retnal mage regstraton approach wth manually selected landmarks and second-order polynomal transformaton model [83] was appled for regstraton of fundus to OCT SLO mage data. Ths regstraton step was necessary to be able to compare outputs of the proposed methodology wth the RNFL thckness at varous postons on the retna Texture analyss Two advanced texture analyss methods were utlzed for feature extracton Gaussan Markov random felds (GMRF) and local bnary patterns (LBP). Markov random felds texture modellng s an effcent tool enablng descrpton of probablty of spatal nteractons n a textural mage so t has been extensvely used n a lot of mage processng applcatons, e.g. n [25], [30], [70], [77], [140], [166], and [171]. Detaled theoretcal aspects about Markov random feld modellng n mage analyss and texture classfcaton can be also found n monographs [90], [137], [145], [178]. The recent development n texture analyss has led to utlzaton of LBP n many mage and vdeo processng applcatons as well. Some of these applcatons can be found e.g. n [53], [91], [99], [128], [149], [155], [158], and [161]. Underlyng theory about the LBP approach can then 7 The regstraton approach was developed as a part of other dssertaton smultaneously proposed by Vratslav Harabš at the DBME, FEEC, BUT

62 be found n [15] and [138]. Both methods were chosen for the proposed methodology because of ther robustness to nose and rotaton- and llumnaton-nvarant propertes Gaussan Markov random felds Ths dssertaton ntroduces the GMRF as a model of the RNFL texture. A set of features s gven by GMRF wth non-causal two-dmensonal autoregressve model. The model assumes the mage texture s represented by a set of zero mean observatons: y ( s), s, { (, j) : 0, j M 1} s, (3.1) for a rectangular M M mage lattce. The ndvdual observaton s then represented by the followng dfference equaton [140]: y ( s) y ( s r ) e( s) r r N, (3.2) S where Ns s a neghborhood set centered at pxel s, r s the model parameter of a partcular neghbor r, and e(s) s a statonary Gaussan nose process wth zero mean and unknown varance. A neghborhood structure depends drectly on the order and type of the model. A ffth-order symmetrc rotaton-nvarant neghborhood structure s assumed, as shown n Fgure 22. Ths neghborhood consders fve parameters depcted by partcular numbers. Fgure 22. A ffth-order symmetrc rotaton-nvarant neghborhood structure. These fve parameters descrbe a relatonshp between the central pxel and ts neghbors. Gaussan varance s the sxth parameter of the model. Fnally, these sx parameters represent features, whch are used for descrpton of the RNFL texture. The least square error (LSE) method s used for estmaton of the model s parameters accordng to the followng equatons [140]:

63 1 T q ( s) q ( s) q ( s) y ( s), (3.3) 1 T 2 ( y ( s) q ( s)), 2 M (3.4) where for an -th-order neghborhood structure. q s) col y ( s r ); 1,..., I r N (, (3.5) Local bnary patterns The LBP method s based on converson of a local greyscale texture nto the bnary code. The local mage area around the central pxel (xc, yc) can be characterzed by the LBP code derved va the equaton [128], [138]: P 1 p LBP ( x, y ) s( g g ) 2, (3.6) P, R c c p c p 0 where 1 ( x) 0 x 0 s. (3.7) x 0 In Equaton (3.6), gc corresponds to grey value of the central pxel (xc, yc) of a local neghborhood and gp(p=0,,p 1) corresponds to grey values of P equally spaced pxels on a crcle of radus R (R > 0) that form a crcularly symmetrc neghborhood structure. Only the sgns of the dfferences s(gp - gc) are consdered to acheve nvarance wth respect to any monotonc transformaton of the mage ntensty (Eq. 3.7). Equaton (3.6) represents a basc rotaton varant verson of LBPP,R operator. Nevertheless, the proposed approach utlzes rotaton-nvarant and unform verson of the basc LBPP,R operator,.e. ru 2 P, R LBP (here superscrpt ru2 means rotaton-nvarant and unform), whch s the most common for many pattern recognton applcatons assumng so-called unform patterns [128]. The unformty of a pattern s formally defned va a unformty measure U of a neghborhood GP [128]:

64 U ( G ) P s( g P 1 g c ) s( g 0 g c ) P 1 s( g P p 1 g c ) s( g p 1 g c ). (3.8) Then, patterns wth a U value of less than or equal to two are consdered as unform. It means these patterns have at most two 0 1 or 1 0 transtons n the crcular bnary code. ru 2 Usng the unformty measure, LBP operator s derved as [128]: P, R ru 2 LBP ( x, P, R c y c ) P 1 s( g g ) p c p 0 P 1 f U ( G ) 2 P. (3.9) otherwse Two optons of LBP were utlzed n the proposed framework. Both optons are based on rotaton-nvarant and unform LBP ru 2 16, 2 operator (.e. P = 16, R = 2). One opton uses only LBP dstrbuton computed from an nput mage. Then, a grey-level hstogram of such parametrc mage s computed and extracton of 6 basc statstcal features follows [59], namely: mean value, standard devaton, skewness, kurtoss, total energy, and entropy. In the second opton, LBP dstrbuton s supplemented wth computaton of local contrast C: 1 P C ( g ), where P 1 g. (3.10) P, R P p p p 0 P p 0 ru 2 Then, n turn, a jont hstogram of LBP and P, R C P, R (LBP/C) s computed. A feature vector s then obtaned from LBP/C jont hstogram by extracton of 14 texture features proposed by Haralck et al. [58] and Othmen et al. [130] (energy, contrast, homogenety, entropy, correlaton, sum average, sum varance, sum entropy, dfference varance, dfference entropy, two nformaton measures of correlaton, cluster shade, and cluster promnence). For completeness, formal descrpton of partcular features s gven n Appendx A Pyramdal decomposton Fnally, a 26-dmensonal feature vector assembled va connecton of partcular texture analyss approaches (GMRF, LPB, and LBP/C) s obtaned. In addton, the features are computed for an orgnal mage resoluton and even for each of the two levels of Gaussan pyramd decomposed mages. Let the orgnal mage be denoted as G0(,j), whch s zero level of the Gaussan pyramd. Then, the l-th level of the pyramd s defned as follows:

65 G, j) w( m, n) G (2 m,2 j n) l ( 1, (3.11) m n where w(m,n) s a two-dmensonal weghtng functon, usually called as generatng kernel. Accordng to [20], recommended symmetrc 5 5 kernel, wrtten n separated l form as w 1 a a,, a,, , where a = 0.4, s utlzed. Fnally, a 78-dmensonal feature vector s obtaned va extracton of the features from G0(,j), G1(,j), and G2(,j). Composton of the fnal feature vector s depcted schematcally n Fgure 23. Fgure 23. Schematc dagram of the fnal feature vector Feature selecton and regresson The am of ths dssertaton s to propose utlzaton of texture analyss n fundus mages for descrpton of changes n the RNFL pattern related to varatons n the RNFL thckness. The ablty of the proposed texture analyss methods, n connecton wth several regresson models, to predct the RNFL thckness has been nvestgated. Dfferent regresson models lnear regresson (LnReg) [108], two types of support vector regresson (-SVR, -SVR) [67], and multlayer neural network (NN) [98] have been tested to predct values of the RNFL thckness usng the proposed texture features. Furthermore, several feature selecton approaches have been nvestgated together wth partcular models. The followng sectons provde a bref ntroducton descrbng underlyng prncples of feature selecton and regresson models used n ths dssertaton Background of feature selecton Feature selecton s wdely used n many areas of data processng, for example n pattern recognton, machne learnng, data mnng, text categorzaton, mage retreval, and genomc analyss. A survey of man applcatons and methods can be found e.g. n [54], [81], and [94]. Fundamental bascs and practcal nformaton to feature selecton can be found also n monographs [41], [55], and [93]

66 Essentally, a feature selecton task should remove rrelevant and/or redundant features wthout affectng learnng performance of a partcular classfcaton/regresson model. Thus, the process of feature selecton generates a new subset of features from the orgnal feature set and reduces ts orgnal dmensonalty. Generally, feature selecton algorthms can have two man components feature search and feature subset evaluaton. Feature search Several feature search methods can be used to fnd desred subset of features n an orgnal feature space. These methods can be categorzed accordng to the startng pont, whch determnes drecton of a partcular search procedure (forward, backward, bdrectonal, random) and a search strategy (exhaustve search, sequental search, random search) [93], [94]. When N s consdered as a number of features, then 2 N s a number of possble feature subsets. Then, an exhaustve search wthn the entre feature space can be used to fnd the optmal soluton. However, an exhaustve search s usually ntractable, when the number of features s large. In ths case, t s more convenent to replace an exhaustve search method by more realstc search strateges and make feature search problem tractable. One of these more realstc technques falls nto the category of sequental search methods that uses so-called greedy hll clmbng (sequental forward selecton, sequental backward elmnaton, and bdrectonal selecton) [41], [93], [94]. Addng one feature per teraton n a partcular drecton, these strateges search for a locally nearoptmal soluton systematcally n each stage tryng to fnd a global optmum. Nevertheless, n many cases, the global optmum s dffcult to fnd. Only local optmum that can be regarded as a suffcent approxmaton of a global optmum s usually dentfed by sequental technques. Another opton can be utlzaton of a random search. As startng pont, randomly selected feature subset s consdered. After that, the process can contnue n two dfferent ways. One way s determnstc, n whch the algorthm contnues wth classcal sequental search technques (e.g. random-start hll-clmbng or smulated annealng). Other way s a completely stochastc process, when each next subset s generated randomly (e.g. Monte Carlo, Las Vegas algorthms). Usually, utlzaton of randomzaton n these approaches can help to overcome the common problem of standard greedy technques wth becomng trapped n local optma wthn the search procedure

67 Feature subset evaluaton Once new subset s selected wthn the search procedure, t needs to be evaluated by a certan evaluaton crteron. Based on whether the evaluaton method depends on the learnng algorthm (model) or not, the feature selecton approaches can be categorzed nto two man categores: flters and wrappers [81], [93], [146]. Flter methods use tranng data alone and do not consder the learnng algorthm that wll be appled for a partcular regresson or classfcaton task. Hence, flter methods has a goal to dentfy relevant and/or redundant features wthout any knowledge of the learnng algorthm. Only some ntrnsc property of the data s utlzed n order to evaluate selected feature subset accordng to some evaluaton crtera. For example, these crtera can nclude measures of dstance between features, nformaton of features, features dependency, or consstency of features [93]. Some of the flter methods utlzng partcularly mutual nformaton and correlaton crtera can be found n [136] and [146]. Wrapper methods wrap the feature search around the learnng algorthm that s used for a partcular data processng [81]. In ths approach, measurement of the model s output for selected feature subset s employed to control searchng for the next subset, untl some stoppng crteron s accomplshed. The stoppng crteron can be e.g. reachng the end of search, reachng some boundary (mnmum number of features or maxmum number of teratons), or reachng requested error rate of a partcular classfcaton/regresson task [93]. Cross-valdaton procedures are usually appled for evaluaton of the model s output n each stage. Generally, n comparson wth flter methods, wrappers can lead for better performance of a partcular learnng model. However, ths s at the cost of hgher computatonal expenses, snce a learnng model needs to be run each tme for evaluaton of each feature subset. Ths also means, the whole feature selecton task s adjusted to a partcular model and t may not work wth another model suffcently. Moreover, sometmes t can result n overfttng of a partcular model. Generally, evaluaton crtera used n wrapper methods can be crtera used commonly for evaluaton of the model performance n varous tasks, for example predctve accuracy for classfcaton or mean squared error of predcton for regresson. An example of the wrapper approach utlzng SVM as a learnng algorthm can be found n [97]. In ths method, a sequental backward selecton strategy s used for feature subset generaton and measurement of classfcaton error to control feature search mechansms

68 Recently, among these two categores of feature selecton approaches, hybrd models combnng the best of both can be found n the lterature [1], [94], [157], [179]. The hybrd models utlze the both from flters and wrappers evaluaton of some ndependent ntrnsc property n data and the model output to control generaton of partcular feature subsets. These models are usually utlzed for tasks that process large amount of data, e.g. for medcal data mnng [1] or dsease classfcaton [179]. In ths dssertaton, flter- and wrapper-based feature selecton approaches were studed and tested. Recent and popular correlaton [146] and mutual nformaton [136] crtera were tested as representatves of flter methods. Then, two sequental search technques (SFS sequental forward selecton and SBS sequental backward selecton) were utlzed as the examples of the wrapper approaches n connecton wth selected regresson models Flter methods Suppose, we have the nput dataset wth N samples and M features X = {x, = 1,, M} and the target varable c. The goal of feature selecton s to fnd from the orgnal M- dmensonal feature space R M a subspace R m of m features whch optmally descrbes c. In other words, the problem s to fnd the subset S wth m features {x} that has the largest statstcal dependency on the target varable c. Then, the popular flter approaches to feature selecton assess the statstcal dependency n term of some metrc, e.g. correlaton [56], [146] or mutual nformaton crtera [136]. In correlaton-based feature selecton (CFS) approach, a smple flter algorthm ranks feature subsets accordng to heurstc correlaton-based evaluaton functon that has been orgnally proposed n [56]: M S =, (3.12) m+m(m 1)r ff where Ms s the heurstc measure of the partcular feature subset S, m s a number of features n the partcular subset S, r cf s the mean feature class correlaton ( f S ), and r ff s the mean feature feature nter-correlaton. The am of CFS approach s to dentfy a feature subset that contans features that are hghly correlated wth the target varable c and uncorrelated wth each other. The algorthm works n an teratve manner searchng for the subset wth the hghest value of MS. Snce an exhaustve search through all possble mr cf

69 subsets s usually ntractable, any of the realstc search strateges can be employed to fnd the best subset of features. In ths dssertaton, a hll clmbng forward feature selecton strategy s utlzed n connecton wth CFS. Other way to fnd a relevant feature subset wth maxmum dependency on the target varable c s utlzaton of mutual nformaton crteron n the mnmum-redundancy- Maxmum-Relevance (mrmr) scheme [136]. Accordng to the concept of mrmr, the maxmum dependency D of the subset S on the target c s expressed n term of maxmal relevance (Max-Relevance): only features wth the hghest relevance to the target c are selected. Ths can be formally wrtten as follows [136]: max S D ( S, c) 1, D I ( x ; c), (3.13) 2 m x S where I ( x ; c) s mutual nformaton computed between the ndvdual features and the target varable. Some of the features selected by the crteron D can be redundant. Ths means, selected feature subset could not be good enough, snce t could contans features that are smlar and statstcally dependent. Therefore, the followng mnmum redundancy (Mn-Redundancy) condton s ncluded n mrmr scheme [136]: mn S 1 R ( S ), R I ( x ; x ). (3.14) 2 j m x, x js The term I x ; x ) n (3.14) represents mutual nformaton computed between ndvdual ( j features n the subset S. Furthermore, the fnal crteron combnes the both constrants D and R, whch results n the complete mrmr approach [136]: max S ( D, R ), D R. (3.15) An ncremental search method s used to fnd the near-optmal feature subset accordng to defned crteron. Frst, the feature wth the hghest value of I ( x ; c) s selected accordng to the crteron D. Then, other features are added n an teratve manner optmzng (3.15), whereas earler selected features reman n a prevous subset. Suppose that -1 features are already selected thus creatng the subset S 1. Then, the next feature x s selected from the set {X S 1 } by maxmzaton of the crteron Φ( )

70 Wrapper methods As dscussed earler, wrapper approaches use output of a learnng model for evaluaton of a partcular feature subset. In each step, the new subset s generated by a certan search strategy and the model s output s evaluated by an evaluaton crteron. Varous search strateges can be used for a subset generaton. Two popular strateges sequental forward selecton (SFS) and sequental backward selecton (SBS) were studed and employed n ths dssertaton. Then, several regresson models, whch are descrbed n the followng secton, were used for predcton of a target varable. SFS starts wth an empty set of features and add one feature per teraton. In each step, each feature canddate that s not so far a part of the current subset s ncluded nto ths partcular subset and the resultng subset s evaluated. At the end of the step, the feature whose addton yelded n the best performance of the current subset s kept n ths subset. Ths way, an algorthm contnues untl a certan number of features s selected, or untl there s no performance mprovement. SBS, n contrast to SFS, starts wth the entre feature set and sequentally removes a feature whose removal yelds the maxmal performance mprovement of the subset. The algorthm contnues untl a certan number of features s left, or untl the performance get too poor. Spearman s rank correlaton coeffcent () and root mean squared error of predcton (RMSEP) are used as evaluaton crtera of models output. can be computed between the model predcted output y and the target varable c as follows [71]: 6 n ( c 2 y ) 1 1, (3.16) 2 n( n 1) where n s number of samples. The values of y and c are separately ranked from 1 to n n ncreasng order. y and c n (3.16) represent the ranks of partcular observatons = 1,,n of the respectve varables. Spearman s rank correlaton coeffcent was chosen because of two man propertes: () t can measure a general monotonc relatonshp between two varables, even when the relatonshp s not necessarly lnear, and () t s robust to outlers due to rankng of values

71 Even when the correlaton between y and c can be strong, the predcted values can stll dffer from the target values wth some error. In order to evaluate model accuracy n the error sense, a frequently used heurstc crteron s utlzed: RMSEP n 1 ( c n y ) 2. (3.17) In each teraton of the wrapper approach (for each feature subset), a crossvaldaton procedure s used to evaluate model output va a chosen evaluaton crteron Regresson models Lnear regresson Lnear regresson model s one of the most wdely used models for regresson, e.g. n statstc and machne learnng applcatons [108]. The response of the model y(x) s a lnear functon of the nput x. Ths can be formally wrtten as follows: where w T x m T y ( x ) w x w x, 1,..., m 1 (3.18) s the dot product between the nput vector x and the model s weght vector w, s the resdual error between the predcted responses and the target values, and m s the number of nput features. The proper weghts of the model are then found by mnmzaton of the resdual errors. Supervsed mult-layer neural network Supervsed mult-layer artfcal neural networks (NN) are wdely utlzed n bomedcal engneerng and other areas especally for data classfcaton, modellng, and predcton [9], [37], [52], [110], [112]. Artfcal neuron elementary unt of NN process an nput data accordng to the general equaton [144]: y m T f ( w x b) f ( w x b), 1,..., m (3.19) 1 where w T x s the dot product between the nput vector x and the neuron's weght vector w, b s the bas, m s the number of nput features, and f s the transfer (actvaton) functon. As can be seen from (3.19), smple artfcal neuron allows makng solutons of the lnear problems only. Then, n contrast to ths smple case, NNs are parallel adaptve systems

72 conssted of more than one artfcal neuron generally allowng solutons of non-lnear problems as well [21]. NN archtecture, type of neuron transfer functons, tranng algorthm, and error calculaton method are generally chosen accordng to the type and the complexty of problem beng solved. In regresson tasks [98], the feed forward NN wth one or two hdden layers s usually used. The number of neurons n NN nput (dstrbutng) layer s defned by the number of features n nput vector. The output layer of such NN nvolves one neuron wth lnear transfer functon whch allows takng real output values. The number of hdden layers and neurons n partcular layers are usually determned emprcally. Durng the tranng process, the NN performance functon (usually defned accordng to the mean of squared errors) s mnmzed by means of solvng an optmzaton problem. Supervsed tranng wth backpropagaton algorthm (BP) [21] s the most commonly used n many NN applcatons. In BP approach, optmzaton s performed va gradent calculaton of the NN performance functon. The weghts and bases are then corrected n the drecton of negatve gradent of the performance functon backward through the NN. Ths conventonal approach, so-called gradent descent approach, s however too slow to solve complex regresson problem [21]. There are faster technques for tranng of NN, such as gradent descent wth momentum or Levenberg- Marquardt algorthm (LM) [21]. The latter s the most commonly used approach based on the numercal optmzaton processes for classfcaton as well as regresson tasks. LM changes the correcton step dependng on the error computed n actual teraton n such a way, that the performance functon (.e. the error) s reduced durng each teraton to reach ts mnmal value. It leads to hgh speed of optmzaton and, as a result, to sgnfcant shortenng of tranng duraton. In ths dssertaton, dfferent archtectures of NN were tested. Fnally, backpropagaton NN wth two hdden layers (ten and fve hdden neurons wth logsgmod transfer functons n the frst and the second hdden layer, respectvely) was utlzed. LM optmzaton algorthm was used for NN tranng. More detal nformaton and mathematcal background of the most popular optmzaton methods used n NN can be found n monograph [79]. Then, full nformaton about the underlyng prncples of dfferent NN approaches, propertes, possbltes, and

73 creatng the NN systems for classfcaton as well as regresson purposes can be found n [13], [21], [36], and [98]. Support-vector regresson Support vector machne (SVM) has become very popular supervsed machne learnng technque that s extensvely utlzed n many areas, e.g. for classfcaton, regresson, as well as other learnng tasks. The concept of SVM was frst ntroduced by Vapnk et al. [167] as a lnear bnary classfer and later extended for non-lnear mult-class problems as well as regresson tasks, so-called support vector regresson (SVR) [35]. An extensve theory and detal descrpton of the prncple of both SVM and SVR can be found n many monographs and papers [13], [19], [65], [67], [108], [150], [151], [167]. In ths secton, frst an underlyng prncpal of classc SVM s presented brefly. Then, the basc concept of SVM s generalzed and extended for regresson purposes. Suppose two-class classfcaton problem, that we have tranng dataset of observaton label pars ( x, ), 1, l, where y, x R n,, and l s the number y 1, 1 of observatons. Then, the goal of the SVM classfer s to construct a hyperplane wth the largest margn that wll be able to separate the postve from the negatve observatons. For observatons that le on the hyperplane, we can wrte [19]: x w b 0, (3.20) where w s the normal vector to the hyperplane and b s the bas parameter. Then, w s Eucldean norm of w and b w s the perpendcular dstance from the hyperplane to the orgn. The margns of a separatng hyperplane are then defned as the shortest dstance from the hyperplane to the closest postve or negatve observaton. The observatons that le on margns of a separatng hyperplane are called support vectors. Durng the tranng of the SVM classfer, a constraned optmzaton algorthm tres to fnd a separatng hyperplane wth the largest margns. To fnd such a hyperplane for lnearly separable cases, all tranng observatons have to satsfy the followng constrants [19]: x w b 1 for y 1, (3.21) x w b 1 for y 1. (3.22) Equatons (3.21) and (3.22) can be further combned nto one set of nequaltes [19]: y ( x w b) 1 0 (3.23)

74 Then, the followng prmal optmzaton problem has to be solved to fnd a separatng hyperplane wth the maxmum margns [19]: mn 1 w, b 2 w 2, (3.24) subject to constrants gven by nequalty (3.23). Equatons (3.23) and (3.24) represent a quadratc programmng problem n whch a functon s mnmzed subject to a set of lnear nequalty constrants. Ths optmzaton problem s usually solved wth help of Lagrangan multplers. The man reasons to reformulate the problem nto Lagrangan representaton are as follows. Frst, solvng the problem n Lagrangan formulaton wll be much easer to handle. Secondly, ths reformulaton allows havng the tranng data n the form of dot products between vectors. The second advantage allows an extenson of the standard form of lnear SVM to handle non-lnear cases. Further nformaton and detals about solvng ths optmzaton problem n Lagrangan form can be found n [13], [19], and [150]. The above descrbed basc prncple of the SVM classfer can be generalzed for more complex learnng tasks coverng non-lnearly separable cases as well as overlappng datasets. In the lterature, a generalzed representaton of the SVM classfer s often referred as so-called soft margn classfer [13], [150]. Then, general constraned optmzaton problem that has to be solved to tran the SVM classfer s formulated as follows [19]: mn w, b, 2 1 w T w C l 1 (3.25) subject to y T w ( ) b) 1, ( x 0, 1,..., l. Functon (x ) n (3.25) serves for mappng the vectors of tranng observatons x nto a hgher dmensonal space by applyng so-called kernel trck, where every dot product s replaced by some kernel functon, wrtten n a general form as K ( x, x ) ( x ) ( x j j T ). Ths procedure s necessary to be able to handle non-lnear classfcaton problems. Dfferent kernels can be used. The most popular kernel functons are mentoned n Table 8 [65]. After the mappng procedure, SVM fnds a lnear separatng hyperplane wth the maxmum margn n a new hgher dmensonal space. Varable C > 0 s the regularzaton (penalty) parameter (3.25) that has to be chosen by

75 the user. A larger value of C results n hgher penalzaton durng optmzaton process. Parameter C thus serves as a trade-off between mnmzng error of classfcaton and maxmzng margn of a separatng hyperplane. Selecton of C s usually made heurstcally utlzng cross-valdaton methods. Parameter s a postve slack varable, whch s nvolved n the optmzaton constrants to be able to tran the model for overlappng datasets. More detals about a generalzaton of SVM classfer can be found n the lterature [13], [19], and [150]. Table 8. The most popular kernel functons used n SVM/SVR. Lnear K ( x, x j ) x T x j Polynomal T d K ( x, x ) x x r, > 0 Radal Bass Functon (RBF) 2 K ( x, x ) exp x x j j j j, > 0 T Sgmod K x, x ) tanh x x r (, > 0, r, and d are kernel parameters. Usually a cross-valdaton procedure s used to fnd the optmal values. j j In the case of classfcaton, the target labels were y 1, 1. In regresson tasks, the problem s slghtly even more general and concerned to estmate real values of the target output 1 z R. The standard form of support vector regresson (-SVR) that has to be optmzed s as follows [67]: mn * w, b,, 2 1 w T w C l C 1 1 l * (3.26) subject to T w ( ) b) z, ( x z T * ( ( x ) b) w,, * 0, 1,..., l. Parameters C > 0 and > 0 have to be chosen expermentally. Parameter s nvolved n the error term to create so-called -nsenstve error zone when constructng a separatng hyperplane. The equaton (3.26) then gves zero error f the absolute dfference between the predcted output and the target varable s less than. The value of can thus specfes desred accuracy of the model and must be defned beforehand

76 The alternatve formulaton of SVR s -SVR, where parameter (0,1] s used to control the number of support vectors [67], [151]. The problem that needs to be optmzed s then as follows [67]: mn * w, b,,, subject to 1 w 2 T 1 w C l l 1 * T w ( ) b) z, ( x (3.27) z T * ( ( x ) b) w,, * 0, 1,..., l, 0. RBF kernels were used n both types of SVR model (-SVR, -SVR) for further testng. The optmal values of the parameters C,, and were dentfed heurstcally through the experments. For more nformaton about -SVR and -SVR formal descrpton and mplementaton possbltes, see [13], [67], and [151]. 3.5 Results and dscusson Evaluaton methodology Evaluaton of the proposed approach for assessment of the RNFL was carred out n two stages. In the frst stage, ablty of the proposed features to predct the RNFL thckness wth partcular regresson models was evaluated, whereas dfferent feature selecton approaches were tested; see an overvew n Table 9. Complete feature set was consdered n method evaluaton as well. A feature vector was computed for each of the 354 ROIs descrbed n Secton The target varable,.e. the vector of the RNFL thcknesses at partcular locatons on the retna, was derved from the nterpolated RNFL thckness map provded by the OCT volume data. Standard repeated random sub-samplng crossvaldaton technque was used for performance evaluaton. Seventy and thrty percent of randomly chosen ROIs were used for tranng and testng the regresson models, respectvely. Whereas, partcular datasets were dsjunctve;.e. each ROI could be ether n the tran or n the test set, not n the both. Ths random sub-samplng procedure was repeated 100 tmes. Spearman s rank correlaton coeffcent and root mean squared

77 error of predcton RMSEP, computed between the predcted output and the vector of RNFL thcknesses, were used to evaluate the models performance. Table 9. An overvew of approaches used for evaluaton of the proposed methodology. Feature selecton approach Complete feature set Flters Wrappers CFS mrmr SFS SBS Regresson models LnReg -SVR -SVR NN Fnal evaluaton crtera RMSEP In the second stage, the proposed method was evaluated utlzng entre fundus mages. Usually, the OCT devce acqures a crcular scan (wth dameter 3.4 mm) around the ONH and the RNFL thckness s then evaluated n ths sngle scan [11]. Hence, evaluaton of the RNFL n fundus mages was performed smlarly as n OCT n a predefned perpapllary area. Frst, the blood vessels n fundus mages were extracted to be able to conduct an analyss n the non-vessel areas only. A crcular scan pattern was placed manually nto the ONH center for each fundus mage. Ths scan pattern conssts of fve partcular crcles (to make the scan reasonably thck). Scannng was performed for ndvdual crcles and the fnal profle was nterpolated by the lnear nterpolaton method that provdes suffcent results. The same nterpolaton technque was used to nterpolate fnal profle n the non-vessel areas as well. Besdes the comparson wth the RNFL thckness provded by the OCT crcular scans, the results of the proposed methodology were compared also wth the straghtforward ntensty crteron that s extensvely used n a subjectve manner by physcans as well as other researchers Evaluaton of the method va cross-valdaton Complete feature set Although dfferent feature selecton technques were studed and tested n ths dssertaton, evaluaton usng all features wthout any feature selecton method was consdered as well. and RMSEP parameters were computed n each run of cross

78 valdaton procedure to evaluate the models output (Fgure 24 and Fgure 25). An averaged values of and RMSEP were computed as shown n Table 10. a) b) Fgure 24. Cross-valdaton results of partcular models usng complete feature set computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram. a) b) Fgure 25. Cross-valdaton results for partcular models usng complete feature set RMSEP computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram. Table 10. Averaged cross-valdaton results of partcular regresson models usng complete feature set. Model [-] RMSEP [µm] LnReg ± ±1.86 -SVR ± ±1.60 -SVR ± ±1.58 NN ± ±1.94 All values of are statstcally sgnfcant wth p values <<

79 Flter CFS Furthermore, two flter-based feature selecton approaches were tested CFS and mrmr. In CFS approach, the best feature subset s dentfed accordng to correlaton crteron (3.16) n an teratve manner. A modfed sequental forward selecton strategy (Secton ) was utlzed for subset generaton. The orgnal verson of SFS stops when addton of the next best feature does not yeld to mprovement of the current subset performance or when a certan number of features s added. The modfed SFS contnues searchng, even when addton of the next best feature results n decrease of a subset performance. Then, ths procedure stops when any other feature s avalable. A subset of 18 features was dentfed usng the CFS approach. and RMSEP parameters were computed n each run of cross-valdaton procedure to evaluate the model output usng the best feature subset (Fgure 26 and Fgure 27). An averaged values of and RMSEP were computed as shown n Table 11. The selected features are numercally lsted at the bottom of ths table. a) b) Fgure 26. Cross-valdaton results of partcular models usng the flter-based CFS approach computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram

80 a) b) Fgure 27. Cross-valdaton results for partcular models usng the flter-based CFS approach RMSEP computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram. Table 11. Averaged cross-valdaton results of partcular regresson models usng CFS. Model [-] RMSEP [µm] LnReg ± ± SVR ± ± SVR ± ± NN ± ± ,4,6,9,11,15,16,17,27,32,39,43,44,57,58,63,77,78 All values of are statstcally sgnfcant wth p values << Flter mrmr The latter flter-based approach mrmr was tested. In ths approach, the best feature subset s dentfed accordng to crteron based of mutual nformaton. As the algorthm proceeds, the features are sequentally added optmzng the crteron (3.15). Ths way, the lst of all ranked features s created n descendng order. A subset of hgh-ranked 12 features was selected from the lst of ranked features wth respect to havng the features from both texture analyss methods equally represented. and RMSEP parameters were computed n each run of cross-valdaton procedure to evaluate the model output usng dentfed feature subset (Fgure 28 and Fgure 29). An averaged values of and RMSEP were computed as shown n Table 12. The selected features are numercally lsted at the bottom of ths table

81 a) b) Fgure 28. Cross-valdaton results of partcular models usng flter-based mrmr approach computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram. a) b) Fgure 29. Cross-valdaton results for partcular models usng the flter-based mrmr approach RMSEP computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram. Table 12. Averaged cross-valdaton results of partcular regresson models usng the flter-based mrmr approach. Model [-] RMSEP [µm] LnReg ± ± SVR ± ± SVR ± ± NN ± ± ,12,63,35,13,73,43,78,11,67,8,6 All values of are statstcally sgnfcant wth p values <<

82 Wrapper SFS Two wrapper-based approaches were tested, dfferentated by search strateges used for a subset generaton SFS and SBS. In SFS strategy, standard forward hll-clmbng procedure s utlzed (Secton ). The procedure starts wth an empty feature set and sequentally adds a feature that yelds n the best mprovement of a partcular subset. Ths proceeds untl there s no mprovement n performance of a partcular feature subset. The RMSEP evaluaton crteron and repeated random sub-samplng cross-valdaton were utlzed for measurement of the model output. Selecton of RMSEP as the evaluaton crteron durng the search procedure was straghtforward, snce t drectly descrbes errors between the predcted output and the target. Ths way, dfferent subsets were selected for partcular models (Table 13). As the best subsets were dentfed, both and RMSEP were evaluated for partcular models. The cross-valdaton results are then presented graphcally n Fgure 30 and Fgure 31, along wth ther averaged values n Table 13. The selected features are numercally lsted below the name of partcular models n ths table. a) b) Fgure 30. Cross-valdaton results of partcular models usng the wrapper-based SFS search strategy computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram

83 a) b) Fgure 31. Cross-valdaton results for partcular models usng the wrapper-based SFS search strategy RMSEP computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram. Table 13. Averaged cross-valdaton results of partcular regresson models usng the wrapper-based SFS search strategy. Model [-] RMSEP [µm] LnReg (5,6,9,11,37,39,48,49,54,64,69,71,78) -SVR (5,6,10,32,37,39,44,58,78) -SVR (5,6,12,32,37,39,44,49,78) NN (1,6,19,40,44,46,78) ± ± ± ± ± ± ± ± All values of are statstcally sgnfcant wth p values << Wrapper SBS A standard backward hll-clmbng technque s used n SBS (Secton ). The algorthm starts wth an entre feature set and sequentally elmnates one feature per teraton whose removal mproves a current subset. The algorthm ends when there s no other feature whose elmnaton would cause better performance than n a prevous step. As n the frst wrapper-based approach, RMSEP crteron and repeated random subsamplng cross-valdaton technque were utlzed for evaluaton of the model performance n each teraton. The results of fnal subsets are shown n Fgure 32 and Fgure 33. Averaged values of fnal crtera evaluated wthn the cross-valdaton are

84 presented n Table 14. Selected features are numercally lsted below the name of partcular models n ths table. a) b) Fgure 32. Cross-valdaton results of partcular models usng the wrapper-based SBS search strategy computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram. a) b) Fgure 33. Cross-valdaton results for partcular models usng the wrapper-based SBS search strategy RMSEP computed between the models predcted output and the RNFL thckness. The results are depcted graphcally n terms of (a) partcular cross-valdaton runs and (b) statstcal boxplot dagram

85 Table 14. Averaged cross-valdaton results of partcular regresson models usng the wrapper-based SBS search strategy. Model [-] RMSEP [µm] LnReg (1,2,3,4,6,8,9,10,11,13,14,15,16,17,20,22, 23,24,25,29,31,33,34,37,40,43,44,48,51,53,54, 58,59,63,64,66,70,72,73,74,77,78) -SVR (5,6,21,22,23,28,29,35,36,37,38,40,43,47,49,51,53, 54,60,68,69,71,72,73,78) -SVR (5,6,19,23,27,28,32,36,40,43,44,48,54,55,57,58, 60,68,69,71,72,73,77,78) NN (6,8,10,11,12,15,17,18,19,20,21,22,26,29,30,32,33,35,37,38,39,40,42,4 3,44,45,46,48,49,50,51,52,53,54,55,57,58,59,60,61,62,63,64,65,66,67,6 8,69,74,75,76,77,78) ± ± ± ± ± ± ± ± All values of are statstcally sgnfcant wth p values << The results of both flter-based approaches are comparable. CFS looks slghtly better than mrmr reachng hgher correlaton and lower error of predcton, but the dfferences between them are not too much expressve. The man advantage of flterbased approaches s that they utlze only ntrnsc propertes of features and dentfed subsets can then be used wth dfferent regresson models. In wrapper-based approaches, feature subset s found for a partcular regresson model and cannot be easly used wth another one. Nevertheless, the cross-valdaton procedure revealed that the results of wrapper-based approaches are better n comparson wth flter-based methods. Hence, adjustment of feature selecton to a partcular model mght be useful for the gven task. Another mportant fndng s that the partcular feature selecton methods dentfed the best subset conssted of features from both texture analyss approaches (GMRF as well as LBP). It shows that features from both approaches are convenent and sgnfcant for descrpton of the RNFL texture. The results of cross-valdaton show that utlzaton of feature selecton methods can mprove performance of partcular regresson models. There s an ncrease of performance n comparson wth complete feature set. However, because the dfferences are small, utlzaton of complete feature set can stll be regarded as benefcal when no lmtatons, e.g. regardng computatonal complexty, are requred

86 3.5.3 Evaluaton of the method usng crcular scan patterns Prevous cross-valdaton stage revealed that the performance of -SVR s slghtly over other approaches so t was consdered for further testng. Moreover, wrapper-based approaches acheved hgher performance than flter-based methods. The cross-valdaton results of -SVR usng SFS and SBS are almost the same and mutually comparable. However, compared to SBS, SFS provdes smaller feature subset wth representatves equally selected from both texture analyss methods. Therefore, the feature set dentfed va the wrapper-based SFS approach s used further. Comparson wth the complete feature set s also ncluded, snce the dfferences between feature selecton and orgnal feature set are not too much expressve. Fgure 34 shows a sgnfcant statstcal relaton between the RNFL thckness and the model traned on the whole dataset of ROIs for complete feature set ( = , RMSEP = µm) and feature subset dentfed va the wrapper-based SFS approach ( = , RMSEP = µm). a) b) Fgure 34. Relaton between the -SVR predcted output and the RNFL thckness for (a) complete feature set and (b) feature subset dentfed va the wrapper-based SFS approach. The model output was computed for each of the 354 ROIs. As descrbed earler, evaluaton s carred out n a dagnostcally nterestng area around the ONH as can be seen from Fgure 35 Fgure 38. Results of the proposed method (Fgure 35b Fgure 38b) are compared wth the RNFL thckness measured by the OCT crcular scans (Fgure 35c Fgure 38c) as well as wth the straghtforward ntensty crteron, whch s usually subjectvely used by physcans and other researchers (thnner RNFL appears darker rrespectvely to the RNFL pattern n fundus mages; Fgure 35a Fgure 38a). Approxmated regresson curves are depcted for each scan showng typcal double-peak crcular scan profles of the RNFL. Evaluaton parameters and RMSEP were computed for each crcular scan extracted from the mages of normal and glaucomatous subjects at the non-vessel areas only (Table 15 Table 16)

87 Table 15. Evaluaton of the method on mages of normal subjects. The values n brackets deal wth approxmated profles (the red curves n Fgure 35 Fgure 36). The values are computed for the non-vessel locatons only. Mnmum and maxmum values are boldfaced n each column. Hghlghted rows denote the mages that are presented n Fgure 35 Fgure 36 (for both mages, the wrapper SFS results are dsplayed). Intensty Predcted mage Image mage Complete feature set Wrapper SFS no. [-] [-] RMSEP [m] [-] RMSEP [m] (0.84) 0.87 (0.94) (12.91) 0.90 (0.98) (10.50) (0.97) 0.76 (0.81) (25.58) 0.81 (0.82) (24.89) (0.64) 0.71 (0.88) (14.99) 0.69 (0.88) (14.59) (0.78) 0.86 (0.98) (16.68) 0.83 (0.98) (16.25) (0.85) 0.89 (0.95) (24.03) 0.85 (0.96) (22.48) (0.93) 0.64 (0.94) (12.14) 0.60 (0.92) (12.75) (0.71) 0.65 (0.91) (11.56) 0.67 (0.91) (11.34) (0.90) 0.74 (0.94) (26.34) 0.82 (0.97) (23.50) (0.81) 0.79 (0.89) (11.71) 0.79 (0.90) (12.02) (0.82) 0.69 (0.90) (19.60) 0.72 (0.92) (17.81) (0.90) 0.90 (0.99) (24.19) 0.90 (0.99) (24.29) (0.80) 0.77 (0.99) (23.40) 0.80 (0.98) (25.49) (0.68) 0.82 (0.92) (22.15) 0.80 (0.92) (22.13) (0.85) 0.79 (0.78) (19.65) 0.79 (0.80) (20.43) (0.77) 0.63 (0.90) (14.45) 0.64 (0.90) (15.18) (0.66) 0.69 (0.89) (21.34) 0.70 (0.90) (20.71) (0.67) 0.68 (0.94) (18.57) 0.71 (0.95) (17.86) (0.78) 0.77 (0.90) (12.33) 0.83 (0.94) (10.58) (0.56) 0.65 (0.99) (12.70) 0.65 (0.99) (13.57) mean 0.59 (0.79) 0.75 (0.92) (18.12) 0.76 (0.93) (17.70) std 0.12 (0.11) 0.09 (0.06) 3.64 (5.21) 0.09 (0.05) 3.48 (5.18) All values of are statstcally sgnfcant wth p values << Table 16. Evaluaton of the method on mages of glaucomatous subjects. The values n brackets deal wth approxmated profles (the red curves n Fgure 37 Fgure 38). The values are computed for the non-vessel locatons only. Mnmum and maxmum values are boldfaced n each column. Hghlghted rows denote the mages that are presented n Fgure 37 Fgure 38 (for both mages, the wrapper SFS results are dsplayed). Intensty Predcted mage Image mage Complete feature set Wrapper SFS no. [-] [-] RMSEP [m] [-] RMSEP [m] (0.41) 0.70 (0.75) (18.53) 0.66 (0.71) (19.49) (0.58) 0.58 (0.67) (17.50) 0.59 (0.68) (18.05) (0.58) 0.54 (0.82) (11.61) 0.57 (0.82) (10.48) (0.03) 0.31 (0.36) (21.50) 0.36 (0.36) (21.30) (0.42) 0.35 (0.45) (30.37) 0.37 (0.41) (28.91) (0.58) 0.68 (0.76) (26.75) 0.69 (0.75) (25.24) (0.31) 0.57 (0.87) (12.32) 0.53 (0.82) (12.91) (0.54) 0.45 (0.47) (19.19) 0.45 (0.43) (18.12) mean 0.38 (0.43) 0.52 (0.64) (19.72) 0.53 (0.62) (19.31) std 0.18 (0.19) 0.14 (0.19) 6.27 (6.47) 0.13 (0.19) 5.86 (6.02) All values of are statstcally sgnfcant wth p values <<

88 The results show that there s a sgnfcant statstcal relaton between the values obtaned va the proposed texture analyss and the RNFL thckness measured by OCT 8. Furthermore, the proposed texture analyss acheved sgnfcantly hgher correlaton than the basc ntensty crteron. Two examples from each table (n tables marked by the grey lne) are shown n Fgure 35 Fgure 38 to demonstrate major outcomes and drawbacks of the proposed approach. Partcularly, Fgure 35 shows results of the mage that acheved one of the hghest performance n terms of along wth one of the lowest error of predcton (mage no. 1 n Table 15) for both, the complete feature set and the SFS wrapper approach. Inspectng the result n detal, one can reveal that the model predcted output follows correctly the RNFL thckness profle wth subtle dfferences only. On the other sde, Fgure 36 shows the mage wth one of the lowest value of and hgher error of predcton (mage no.6 n Table 15). Ths can be caused probably by varaton n mage qualty (blurrng and presence of nose n a couple of mages). Other drawback concerns the blood vessels that cover rather large area of the retna, especally n the ONH surroundngs. At the locatons of the blood vessels and ther near neghborhood, the texture representng the RNFL s mssng n fundus mages. Hence, the texture analyss s demsed to be carred out at the locatons wthout the blood vessels only. Due to ths ssue, the predcted values are reduced partcularly at locatons of the major blood vessel branches. However, even n the worst case, the evaluaton revealed that the results are stll relevant capturng varatons n the RNFL thckness sgnfcantly. Fgure 37 and Fgure 38 then show the best and the worse examples for glaucomatous subjects, respectvely. The performance of the method evaluated usng mages of glaucomatous subjects s lower than for normal subjects. Generally, ths s probably due to worse mage qualty of glaucomatous subjects that were tested (possbly caused by cataracts and unclear ocular meda). In addton, lmted number of patents also nfluences the evaluaton. However, despte the drawbacks mentoned, there s a sgnfcant mprovement aganst the common ntensty crteron for the mages of both the normal as well as glaucomatous subjects. The evaluaton part revealed that the proposed methodology could suffcently contrbute to the RNFL assessment based only on fundus camera. In comparson to the ntensty, the proposed texture approach s able to capture contnues varatons n the RNFL thckness and thus can be used for possble detecton of the RNFL thnnng caused by pathologcal changes n the retna. Addtonal advantage of 8 Sgnfcance of the results was statstcally valdated by t-test at the 5% sgnfcance level

89 ths texture approach s that the proposed features are nvarant to changes of llumnaton and lght reflecton. a) b) c) Fgure 35. Images of crcular scans of the normal subject no. 1 and correspondng profles: a) orgnal GB fundus mage wth ntensty profle, b) model predcted output wth correspondng profle, c) SLO mage wth crcular scan pattern and the RNFL thckness profle. Red curves represent polynomal approxmaton of each profle. The red arrow ndcates drecton of scannng. a) b) c) Fgure 36. Images of crcular scans of the normal subject no. 6 and correspondng profles: a) orgnal GB fundus mage wth ntensty profle, b) model predcted output wth correspondng profle, c) SLO mage wth crcular scan pattern and the RNFL thckness profle. Red curves represent polynomal approxmaton of each profle. The red arrow ndcates drecton of scannng