CHAPTER - 2 LITERATURE REVIEW

Transcription

1 CHAPTER - 2 LITERATURE REVIEW Currently, there is an increasing interest for establishing automatic systems that screens a huge number of people for vision threatening diseases like diabetic retinopathy and to provide an automated detection of the disease. Image processing is now becoming very practical and a useful tool for diabetic retinopathy screening. Digital imaging offers a high quality permanent record of the fundus images, which can be used by ophthalmologists for the monitoring of progression or response to the therapy. Digital images have the potential to be processed by automated analysis systems. Fundus image analysis is a complicated task, because of the variability of the fundus images in terms of colour/gray levels, the morphology of the anatomical structures of the retina and the existence of certain features in different patients that may lead to a wrong interpretation. In the literature, numerous examples of the application of digital imaging techniques used in identification of diabetic retinopathy can be found. There have been few research investigations to identify retinal components such as blood vessels, optic disk, fovea and retinal lesions including microaneurysms, hemorrhages, and exudates in the literature [13-17]. The major contributions to extract the normal and abnormal features of fundus images are described in this chapter.

2 This chapter is organized as follows. In Section 2.1, the literature corresponding to blood vessel segmentation is reviewed. The major works related to localization and contour detection are discussed in Section 2.2. Section 2.3 reviews the literature corresponding to detection of macula and fovea. In Sections 2.4 and 2.5, the background information of bright lesions and dark lesions is presented RETINAL BLOOD VESSEL SEGMENTATION Several studies were carried out on the segmentation of blood vessels in general, however only a small number of them were associated to retinal blood vessels. In order to review the methods proposed to segment vessels in retinal images, seven classes of methods have been considered: matched filters, vessel tracking, morphological processing, region growing, multiscale, supervised and adaptive thresholding approaches Matched Filters Matched filters were based on a correlation measure between the expected shape sought for and the measured signal. The algorithm presented by Chaudhuri et al. [18] was based on directional 2D matched filter. To enhance retinal vasculature a 2D matched filter kernel was designed to convolve with the original fundus image. The kernel was rotated into either eight or twelve orientations to fit into

3 blood vessels of various configurations. A number of kernel shapes have been investigated. Gaussian kernels were used in [18-20]. Kernels based on lines [21] and partial Gaussian kernels were also used [22]. A number of approaches were also proposed to identify true blood vessels from the matched filter response. A region based threshold probe was applied to the response of matched filter to segment the vessel network [20]. An amplitude modified second order differential Gaussian filter was proposed by Gang et al. [23] to detect the vessel network at different scales that match their widths. This was achieved by changing the amplitude, so that responses can be merged over scales. Local entropy based thresholding was proposed by Chanwimaluang et al. [24] Tracking Methods The tracking methods look for a continuous blood vessel fragment starting from a point given either manually or automatically, depending on certain local information [25-31]. These methods normally try to get the path which best matches a vessel profile model. Sobel edge detectors, gradient operators and matched filters were used to find the vessel direction and boundary. Even though these methods were puzzled by blood vessel bifurcations and crossings [32-33], they can yield precise measurements of blood vessel tortuosity and widths.

4 Morphological Processing To segment the blood vessels in a retinal image, mathematical morphology can be used since the vessels were the patterns that exhibit morphological properties such as connectivity, linearity and curvature of vessels varying smoothly along the crest line. But background patterns also fit such a morphological description. In order to discriminate blood vessels from other similar structures, cross - curvature evolution and linear filtering were employed by Zana et al. [34]. A two step method was applied to segment vasculature by Fang et al. [35]. Firstly, mathematical morphology filtering coupled with curvature evolution was utilized to enhance the blood vessels in fundus images. The major limitation of this technique was that significant features like intersection and bifurcation points may be missed out. To recover the complete vessel network a reconstruction procedure employing dynamic local region growing was performed Region Growing Approaches In region growing approaches, it was considered that, pixels which were nearer to each other and possessing similar gray level intensities were expected to fit into the same object. These approaches recruit pixels incrementally in a region starting from a seed point based on predefined criteria [36-38]. The criteria used for segmentation were value similarity and spatial proximity. The limitation of region growing approaches was they often need user supplied seed points. Region

5 growing may yield over segmentation because of the changes in image intensities. Thus, the segmentation result was often post-processed in these approaches Multiscale Approaches Vessel segmentation in multiscale approaches was performed by varying image resolutions [39-45]. The main advantage of using these approaches was their efficient processing speed. In these approaches larger blood vessels were segmented from regions having low resolution and finer vessels were segmented from regions having high resolution. Multi-scale analysis of first and second order spatial derivatives of the gray level intensity image was used for the segmentation of blood vessels having different widths, lengths and orientations [40]. A two step region growing procedure was applied in this method. The growth was constrained to regions with low gradient magnitude in the first step. This constraint was relaxed in the second step to permit the borders between regions to be defined. A vascular modeling algorithm was proposed by Wang et al. [45] based on a multiresolution image representation. A two dimensional Hermite polynomial was used over a range of spatial frequency resolutions to model the retinal vasculature in a quad-tree structure. The parameters of the Hermite model were estimated using expectation maximization type optimization technique upon which the information based process was then employed to select the most appropriate scale or model for modeling each region of the image. The

6 vascular network was segmented by Softka et al. [46] based on the response of multiscale matched filters, gradient at the boundary of blood vessels, the edge strength at the boundary and blood vessel confidence measure Supervised Methods Recently, several supervised methods [47-49] focusing on 2D fundus images were explored to obtain better results. Two blood vessel detection methods in digital fundus images based on line operators were developed by Perfetti et al. [47]. The response of the line detector was thresholded to attain pixel classification which was unsupervised in the first segmentation method. In the second segmentation method, a feature vector comprising two orthogonal line detectors and target pixel s gray level was used for supervised classification. A pixel based classification method was developed to segment blood vessels [48]. This method classified each pixel of the gray level fundus images as blood vessel component or non-blood vessel component, based on feature vectors of the fundus image. The responses of two dimensional Gabor wavelet transform attained at multiple scales and intensity at each image pixel constitute the feature vectors. Then, Bayesian classifier with Gaussian mixture model was trained to classify these feature vectors. Image ridges were extracted by Staal et al. [49] and applied to compile primitives in the form of line elements. By assigning each image pixel of the fundus image to the closest line element, the fundus image was partitioned into patches. For every line element a

7 local coordinate frame was constituted. The feature vectors of the line elements were computed using the properties of the patches. Nearest neighbourhood classifier was employed to classify these feature vectors using sequential forward feature selection Adaptive Thresholding Methods The features of the vasculature were captured by using nonlinear orthogonal projections in Zhang et al. [50] and a local adaptive thresholding algorithm was employed for vessel detection. Knowledgeguided adaptive thresholding was employed by Jiang et al. [51] to segment vessel network. Multi-threshold probing was applied to the fundus image through a verification procedure that makes use of a curvilinear structure model. The relevant information about objects, including shape, colour/intensity, and contrast was incorporated, which guides the classification procedure. The methods discussed above for blood vessel segmentation can work well to segment the major parts of vasculature. However, the major challenges confronting the above vessel segmentation methods are: Extraction of thinner blood vessels as the image contrast is generally low around thin vessels; The presence of lesions as they may be mis-enhanced and mis-detected as blood vessels. In order to solve these challenges, a new method for vessel

8 segmentation is proposed in this thesis. The method uses the intensity of red and green channels and the thresholding based on relative local entropy with histogram compression and translation LOCALIZATION AND CONTOUR DETECTION OF OPTIC DISK Reliable and efficient optic disk localization and contour detection are significant tasks in an automated diabetic retinopathy screening system. Optic disk localization is required as a prerequisite for the succeeding stages in many algorithms applied for identification and segmentation of the anatomical and pathological structures in retinal images. Accurate localization of the optic disk and its contour are very useful in detecting proliferative diabetic retinopathy. Because of the circulation problems occurred during the early stages of diabetic retinopathy, new blood vessels which are very delicate and weak will be developed largely in the optic disk region of the retina. If the location of optic disk is known, then the position of other regions of medical significance like macula and fovea can be determined. The location of optic disk can be employed as a marker for retinal image registration. As the optic disk or optic nerve head is the source of major retinal vessels, its centre may be employed as a beginning point for vessel tracking approaches. Many schemes have been proposed to localize optic disk. Majority of these schemes were finding only the location of the optic disk and not addressing the

9 problem of contour detection of optic disk. Accurate localization of optic disk is surprisingly complicated, because of its highly variable appearance in fundus images Localization of Optic Disk The algorithms [52-54] localize the optic disk by finding the largest cluster of image pixels having high intensity values. The area having highest gray level intensity variation of neighbouring image pixels was identified as optic disk [14]. The variation in intensity of neighbouring image pixels was evaluated by using 80 x 80 sub-images. The point having the largest variation of intensity was marked as the location of optic disk. These algorithms did not consider the retinal images having bright lesions. However, retinal images with small lesions have been considered by Lalonde et al. [54]. The method proposed by Hoover et al. [55] uses the convergence of the vascular tree as the vital feature for optic disk detection. In this method, the optic disk was identified as the focal point of the vascular tree. The convergence of the vascular tree was detected by finding the end points of the linear shapes such as blood vessels. A combination of mathematical morphology and watershed transformation was used to detect optic disk [56]. In this method, a shade correction technique was used to decrease the contrast of hard exudates and to eliminate the slow background variations. Local gray level intensity variance of neighbouring pixels was applied to the shade corrected image to estimate the locus of the optic disk. Watershed transformation was

10 used to locate the optic disk boundaries. Osareh et al. [57] proposed template based optic disk localization. This approach employed colour normalization of retinal images followed by template matching. A hybrid approach was used to localize optic disk based on intensity and vessel structure based features [58]. First, the candidate optic disk locations were derived based on curvature information that was used to detect hill type topographical feature which inherently encodes intensity features. Each candidate was assigned a confidence measure that was derived using vessel structure information. Consequently, a candidate location possessing the highest value was considered as optic disk. A two step strategy was used to locate optic disk [59]. Firstly, the candidate optic disk regions were chosen by determining all the bright regions in a local surrounding. Secondly, skeleton map of the binary blood vessels was obtained using the scheme proposed by Zhang et al. [60]. In this method, Fractal analysis was employed at the candidate optic disk regions. The candidate optic disk region having the maximum fractal dimension was selected as optic disk. When compared to other bright regions, optic disk region will have the highest fractal dimension because in optic disk region all the major blood vessels will be merged. The algorithms proposed by Tamura et al. [52], Liu et al. [53] and Sinthanayothin et al. [14] fail to localize the optic disk when large bright lesions coexist in a fundus image. Distinguishing the optic disk from large exudates became a challenge. These methods obtained an acceptable result only in normal fundus images in which the optic

11 disk region was bright and observable. The algorithm proposed by Osareh et al. [57] assumed that the optic disk is approximately circular and consisting of bright pixels. This algorithm failed when optic disk is not the biggest and the brightest region in the retinal image. To localize the optic disk accurately, a new scheme is proposed in this thesis based on blood vessel information in the optic disk region. This method localizes the optic disk by determining the vessel branch having most number of blood vessel connections Contour Detection of Optic Disk The contour of the optic disk is helpful to study the progress and treatment results of eye diseases like diabetic retinopathy and glaucoma. As the profile of the optic disk is like oval or round, the contour of the optic disk was identified as an ellipse or a circle [52-54]. A two dimensional Hough transform method was applied to detect the contour of the optic disk [52-53]. Edge detection techniques were used in this method to identify the estimated circle of optic disk. The optic disk contour was detected using Hausdorff-based matching among the edges detected and circular templates having different sizes [54]. Estimating the contour of the optic disk as an ellipse or a circle will not give adequate information for physicians. Since the contour or boundary of the optic disk is very important to analyse vision

12 threatening diseases like diabetic retinopathy and glaucoma, the accurate contour of the disk has to be determined. To identify the accurate boundary of the optic disk, snakes or active contours were employed [61-63]. The foremost benefit of using these methods was their ability to bridge the discontinuities in the contour of the optic disk being located. The major hurdle to apply snakes for optic disk contour identification was how to eliminate the influence of thick retinal blood vessels in the optic disk region. Another disadvantage of above mentioned methods was that they require manual initialization. The optic disk boundary was segmented in two stages [63]. Firstly, the original retinal image was preprocessed based on local minima detection and mathematical morphology in order to eliminate the interfering vessels and detect the contour of the optic disk accurately. To eliminate the influence of vessels in the optic disk region, gray level dilation was performed. Next, erosion is performed to restore the boundaries to their initial position. A structuring element of 5 x 5 pixels was used. The optic disk contour was located using deformable active contours method. The snake or active contour was based on an external image field known as Gradient Vector Flow (GVF). This method was evaluated on a dataset set containing 9 fundus images and the author reported accurate optic disk boundary localization in all the test images. The algorithms proposed by Viranee et al. [64] and Osareh et al. [57] also used GVFs to detect the contour of the optic disk. A modified active shape model was presented by Li et al. [65] to detect the contour of optic disk. In this method, training data was

13 used to conceive a point distribution model. In order to locate the occurrence of shapes similar to optic disk, an iterative searching method was applied to the new fundus image. For retinal images having ill-defined optic disk and for images having fuzzy elliptic disk, algorithms such as 2D Hough transform [52] and GVF snake methods [57, 63, 64] failed to detect the contour of the optic disk. A new method based on Geometric active contours is presented in this thesis to identify the contour of the optic disk. The proposed method uses mathematical morphology in Lab space and Geometric active contours with new variational formulation to identify the contour of optic disk. This method works accurately even though the boundary of optic disk is not continuous or blurred DETECTION OF FOVEA The position of an abnormality relative to the location of fovea is useful for effective diagnosis of diabetic retinopathy and other retinal diseases. Fovea is the centre of macula and is present at approximately 2.5 times the optic disk diameter from the optic disk [14]. The macula is commonly a hazy darker area than the surrounding retinal tissue. To detect macula and fovea, template matching approach was used by Sinthanayothin et al. [14]. The template was a Gaussian blob. A model based approach was used to detect fovea [65]. In this approach, information derived from an active shape model was employed to identify fovea. A single point

14 distribution model was utilized to detect fovea [66]. This method used a cost function that depends on grouping of global and local cues to locate the exact position of the model points. An appearance-based localisation method using different image channels was applied to detect fovea [67]. The method proposed by Sagar et al. [68], first finds the vessel pixels and then detects macula by finding the darkest cluster of pixels near the optic disk. A Novel approach to detect fovea is developed in this thesis based on the information regarding blood vessels and optic disk. This approach is robust to the occurrence of abnormalities and low illumination areas that appears similar to fovea BRIGHT LESION DETECTION Among abnormalities caused by diabetic retinopathy, bright lesions are one of the most usually occurring lesions. The bright lesions are due to the damaged blood vessels which leak proteins and lipids. During the progression of diabetic retinopathy, the size and distribution of bright lesions may be changed. The detection and quantification of bright lesions will considerably contribute to the mass screening and estimation of back ground diabetic retinopathy. Here, the major bright lesion identification methods in the literature are reviewed. Philips et al. [69-70] used a two step strategy to segment bright lesions. Firstly, colour retinal images were shade corrected to eliminate the non-uniformities. Secondly, the contrast of bright lesions was improved. To segment the bright lesions from the retinal

15 images, global and local threshold values were used. This method had reported a lesion-based sensitivity in between 61% and 100% (mean 87%) [70]. The algorithm proposed by Ward et al. [71] also used a two step strategy. First the fundus image was pre-processed to decrease shade variations in the background and to improve the contrast between the image background and the bright lesions. The bright lesions were segmented from the background on a brightness or gray level basis. This algorithm needed user involvement for selecting the threshold. A dynamic thresholding algorithm was developed to segment exudates from retinal images [53]. The retinal images were first divided into 64 x 64 pixel patches. Using the histogram of each patch, a local threshold was calculated. The dynamic threshold of every image pixel was calculated using the interpolation of local thresholds of four neighbouring patches which contain that image pixel. A prototype was presented by Goldbaum et al. [13] on automated diagnosis and retinal image understanding. Features like compactness, object colour, border colour, texture measures, edge gradient, area and turns per length of the border were used to segment bright lesions. This method detected bright objects with an accuracy of 89%. Ege et al. [72] used template matching, region growing and thresholding methods to detect bright lesions. Before applying these techniques, the retinal images were preprocessed using median filter to eliminate noise. Using Bayesian classifier the bright lesions were

16 then classified into exudates, cottonwool spots and noise. This classifier achieved accuracies of 62% for exudates and 52% for cottonwool spots. A minimum distance discriminant classifier was used by Wang et al. [73] to classify each pixel into bright lesions (hard exudates, cottonwool spots) or non-lesions (vessels and background). For image based evaluation, this approach got 100% sensitivity and 70% specificity. In addition to the discussed techniques above, neural networks were also exploited to classify the retinal abnormalities in a few studies. Gardner et al. [74] divided the retinal images into sub-images of size 20 x 20. Subsequently, these sub-images were applied to a back propagation neural network that was trained for five days with 400 inputs. This method detected the vessels, hard exudates and hemorrhages. The sensitivity of hard exudate detection technique was 93.1%. This performance was the outcome of classifying the image into 20 x 20 pixel patches. Neural networks were used by Hunter et al. [75] to classify bright lesions. In this method, the retinal image was divided into 16 x 16 sub-images and eleven inputs were applied to train the neural network. The neural network was intended to discriminate the exudates from drusen and achieved 91% lesionbased performance. A recursive region growing algorithm was used to segment exudates in fundus images [76]. A sensitivity of 88.5% and specificity of 99.7% were reported. However, these performances were measured based on 10 x 10 patches. Gray level variation of the exudate

17 candidates in the green channel was used to segment exudates [77]. After initial localization, using mathematical morphology techniques, the contours of the exudates were subsequently determined. The approach used three parameters for detecting exudates. Size of the local window that was used to compute the local variation of pixels. First threshold that was used to find the candidate exudate regions. The second threshold that represents the minimum value by which a candidate bright lesion must differ from its neighbouring background image pixels to be classified as exudates. The mean sensitivity and mean predictivity achieved by this method were 92.8% and 92.4% on a dataset of 15 abnormal fundus images. The method proposed by Osareh et al. [78] first normalizes the fundus image using histogram specification. Local contrast enhancement was performed to improve both the contrast of lesions against the background and the overall colour saturation. This was followed by Fuzzy C-Means (FCM) clustering to segment probable exudate candidates. Multilayer perceptron neural network with ten inputs was used to classify the exudate candidates from nonexudates. This method attained a sensitivity of 92% and specificity of 82%. Same authors used SVMs to classify the exudate candidates from non-exudates and yielded a sensitivity and specificity of 87.5% and 92% respectively [79]. The algorithm proposed by Sopharak et al.

18 [80] also used FCM clustering to segment exudates for non-dilated retinal images. They used four dominant features hue, standard deviation, intensity and adaptive edge by FCM to get coarse segmentation followed by fine segmentation using morphological reconstruction. Zhang et al. [81] presented a three stage approach to detect bright lesions and classifying them into exudates and cottonwool spots. Firstly, local contrast enhancement was applied as a preprocessing stage. An Improved Fuzzy C-Means was applied in Luv colour space to extract all the candidate bright lesions. A hierarchical SVM classification was used to classify bright lesions from non-lesions. The authors also classified exudates and cottonwool spots using a polynomial kernel in the SVM classification. This method classifies bright lesions and bright non-lesions with a sensitivity and specificity of 97% and 96% respectively. In classifying exudates and cottonwool spots this method got a sensitivity of 88% and specificity of 84%. The algorithm proposed by Li et al. [65] divides the retinal image into 64 sub-images and exudate detection was performed in each subimage. Region growing and edge detection techniques were applied to detect the exudates. The sensitivity and specificity for detecting exudates were 100% and 71% respectively. Image contrast was firstly enhanced by means of a neurofuzzy subsystem, where properly codified fuzzy rules were realized using a sparsely connected 4x4 cell Hopfield neural network [82]. Enhanced contrast images were then properly segmented to isolate suspect areas

19 in binary output images after computing the optimally global threshold by a neural network based subsystem. To detect bright lesions in fundus images, a new method based on Spatially Weighted Fuzzy C-Means (SWFCM) clustering is proposed in this thesis. The weight in the SWFCM algorithm is inspired by KNN classifier that considers the influence of neighbourhood on the central pixel, is changed to improve the performance of clustering. Due to the consideration of the neighbourhood information, the proposed method is noise resistant. The gray level histogram of the fundus image is used in the proposed SWFCM clustering instead of the whole data of the image. Hence the computation time is very less for the proposed method compared to other FCM based methods discussed above DARK LESION DETECTION Microaneurysms and Hemorrhages are the red or dark lesions found in the retinal images. Microaneurysms appear in the very early stages of diabetic retinopathy and hemorrhages appear in the proliferative diabetic retinopathy stage. Hence, detection of former tells us detect the disease at the earliest and later tells whether diabetic retinopathy is in advanced stage or not. For this reason, the detection of these two dark lesions is very important. Microaneurysm and hemorrhage counts are very good indicators of progression of the disease.

20 Several methods for detecting dark lesions were reported in the literature. The dark lesion detection algorithm proposed by Marino et al. [83] contains three stages: firstly, a set of correlation filters were applied to extract candidate dark lesions. In the second stage, segmentation based on region growing was applied to reject candidate dark lesions whose size does not fit in the pattern of dark lesion. Finally, three tests i.e. a shape test, an intensity test and a test to eliminate the points that fall inside the blood vessels (only lesions outside the vessels were considered) were used to find true dark lesions. A nonlinear curve with brightness values of the HSV space was used to change the brightness of the fundus image [84]. Gamma correction was employed to emphasize brown regions on each red, green and blue-bit image. Density analysis was applied to detect candidate hemorrhages. Finally, a rule based method and three Mahalanobis distance classifiers were employed to eliminate the false positives. Gray level grouping based contrast enhancement was applied to improve the contrast of the green channel [85]. Then candidate dark lesions were extracted by employing automatic seed generation. Spatio-temporal feature map classifier was used to classify true dark lesions from non-dark lesions. Abhir et al. [86] had applied an orientation matched filter to the preprocessed retinal image. The output of orientation matched filter was thresholded to obtain a set of potential candidates. Eigen image

21 analysis was used to eliminate certain noises that resemble the shape profile of microaneurysms. Finally a second threshold was applied on the Eigen-space projection of the candidate regions to remove the false positives. A novel dark lesion detection method was presented by Niejmer et al. [87] based on pixel classification and morphology based segmentation. In pixel based classification, vasculature and dark lesions were separated from the background of the image using KNN classifier. The dark lesion objects were classified using extensive number of features and a KNN classifier. Microaneurysms and hemorrhages were treated as holes and morphological filling was performed on the green channel to identify them [88]. The unfilled green channel image was then subtracted from the filled one. The resultant image was thresholded to get an image (R) having microaneurysm patches. To remove noisy vessel segments, the full blood vessel network skeleton was dilated and subtracted from the image R. The remaining patches were further classified using intensity properties and a colour model based on the detected blood vessels. Candidate microaneurysms were detected by taking the Maximum of Multiple Linear Top-Hats (MLTH) applied to the inverse image [89]. MLTH was adapted to detect larger objects like hemorrhages at multiple scales by repeating with multiple structuring elements [90]. Later the candidate hemorrhages were classified by using SVM classifier.

22 A generalized Eigen vector was applied to get the locations of microaneurysms [91]. The probable locations of the microaneurysms were found by determining the position of the highest absolute value of the second smallest Eigen vector. These locations were analyzed using specific features of microaneurysms to identify the true microaneurysms. A four step strategy was employed to extract microaneurysms [92]. Firstly, local contrast enhancement was used for preprocessing. Using the definition of bounding box closing, small details were extracted. An automatic threshold depending on image quality was calculated. Finally, false positives were eliminated. The major challenges in dark lesion detection for the algorithms discussed above are: Segmentation of small microaneurysms in the areas of low image contrast; and The presence of bright pathologies. Normally bright lesions have sharp edges. When these bright lesions are close to each other small islands of normal retina are created between them. They may be picked up as false positives. To solve these problems, a hybrid dark lesion detection method is proposed. This method combines morphological based dark lesion detection and candidate dark lesion detection scheme based on matched filtering and local relative entropy.