Relevance of Computational model for Detection of Optic Disc in Retinal images

Transcription

1 Relevance of Computational model for Detection of Optic Disc in Retinal images Nilima Kulkarni Department of Computer Science and Engineering Amrita school of Engineering, Bangalore, Amrita Vishwa Vidyapeetham (University), India Amudha J. Department of Computer Science and Engineering Amrita school of Engineering, Bangalore, Amrita Vishwa Vidyapeetham (University), India Abstract Optic Disc (OD) detection is a key pre processing component in many algorithms designed for automatic extraction of retinal anatomical structures and lesions thus, reliable and efficient OD localization is significant tasks in ophthalmic image processing. We aim to investigate the relevance of computational saliency model in retinal images in the context of OD detection. The optic disc in a retinal image is characterized as high contrast region. This approximately corresponds to disjunctive targets in a visual search task. Saliency maps were computed using popular model Itti-Koch. Then mathematical morphology and Otsu s algorithm used on the saliency map for OD detection. The resulted images satisfactorily showed OD detection. The results are promising and give new insight to the use of the saliency map for medical images. 1. Introduction The OD is considered one of the main features of a retinal image, where methods are described for its automatic detection. It is a key pre-processing component in many algorithms designed for the automatic extraction of retinal anatomical structures and lesions thus, reliable and efficient OD localization is significant tasks in ophthalmic image processing. The process of detecting the OD aims only to correctly detect the centroid of the OD. On the other hand, disc boundary detection aims to correctly segment the OD by detecting the boundary between the retinal and neuro-retinal rim. The automatic and efficient detection of the position of the OD in color retinal images is an important and fundamental step in the automated retinal image analysis system [23], [24]. To successfully find abnormal structures in a retinal image, it is often necessary to mask out the normal anatomy from the analysis. An example of this is the OD, an anatomical structure with a bright appearance, which should be ignored when detecting bright lesions. The attributes of OD are similar to attributes of hard exudates in terms of color and brightness. Hard exudates are among the preliminary signs of diabetic retinopathy, a major cause of vision loss in diabetic patients. Therefore OD is located and removed during hard exudates detection process and developing automated screening systems for diabetic retinopathy and glaucoma avoiding false positives. In case of diabetic maculopathy lesions identification, masking the false positive OD region leads to improvement in the performance of lesion detection. In this paper we aim to investigate the performance of computational saliency models to detect the OD in retinal images. Saliency intuitively characterizes some parts of a scene which could be objects or regions that appear to an observer to stand out relative to their neighbouring parts. The area of the image attended to visually is deemed as salient. The term salient is often considered in the context of bottom-up computations [8], [9]. Computational Visual Attention (CVA) is an artificial intelligence for simulating this biometric mechanism. With this mechanism, the difference feature between region centre and surround would be emphasized and integrated in a conspicuity map. Computational vision systems face the same problem as humans as there is a large amount of information to be processed. The term attention is common in everyday language and familiar to everyone. Attention is a general concept covering all factors that influence selection mechanisms, whether they are scene driven bottom-up (BU) or expectationdriven top-down (TD). Visual attention is a mechanism in human perception which selects relevant regions from a scene and provides these regions for higher- 1896

2 level processing as object recognition. This enables humans to act effectively in their environment despite the complexity of perceivable sensor data. The basis of many attention models dates back to Treisman and Gelade s [10] Feature Integration Theory, where they stated which visual features are important and how they are combined to direct human attention over pop-out and conjunction search tasks. Koch and Ullman [9] then proposed a feed-forward model to combine these features and introduced the concept of a saliency map which is a topographic map that represents conspicuousness of scene locations. They also introduced a winner-take-all neural network that selects the most salient location and employs an inhibition of return mechanism to allow the focus of attention to shift to the next most salient location. While FIT [10] explains the saliency of different locations of a visual input, what makes these locations salient is partially addressed by visual search paradigms. Visual Search is a common human activity. Searching for a friend in a crowd is an everyday example. It is also important in diagnosing diseases as radiologists search for lesions and other abnormalities in medical images before writing reports. Finding salient regions in an image is of interest to computer vision as well since visually salient features in an image are generally invariant to many image transformations and carry important image information [11]. Attempts have been made to use saliency to address problems such as object detection [12], image compression [13], tracking and image retrieval [14]. Several psychophysical and computational models of visual attention have been proposed in the literature. Their main objective is to simulate the behavioral data and to better understand human perception. Most of these models have been studied and validated in the context of viewing natural scenes. A visual search task involves identifying targets from surrounding distracters. The goal of visual search in medical images is generally one of evidence gathering about the possibility of any `abnormality' in the condition of a patient. The search therefore involves disregarding the `usual' and detecting the `unusual' visual elements. This is akin to searching for anything that is out of ordinary in a given face image. In medical images, abnormalities are wide ranging in terms of appearance. They can be glaring to subtle. We aim to investigate the performance of computational saliency models in medical images in the context of abnormality detection. Specially, we are interested in investigating whether the computational saliency models can detect OD in retinal images. The most efficient of searches are those in which a single basic feature (e.g. orientation) dense the target and distracters are homogeneous. Such targets are called disjunctive targets. In general, pre-attentive visual processes are sufficient to identify such targets, which is called pop-out phenomenon [10, 15]. If targets and distracters share common features, search becomes inefficient and focused or serial attention is required to identify such targets [16]. Such targets are called conjunctive targets. Figure 1 shows sample search tasks involving disjunctive and conjunctive targets. In stimulus shown in figure 1(a), identifying target (`X in red color)') pops-out among distracters (`X in black color') and can be identified by pre-attentive processes. Such targets are called disjunctive targets. In figure 1(b), identifying orange square (target) among blue squares and orange triangles requires serial attention and such targets is called conjunctive targets. Figure 1: Sample stimulus showing (a) disjunctive and (b) conjunctive target searches. The OD is an anatomical structure with a bright appearance. Following the above-mentioned concepts, we chose OD detection in retinal images correspond to conjunctive and disjunctive targets in a visual search (see figure 2 a) to study the relevance of computational visual saliency in medical images. The hard exudates (HE) in a retinal image (see figure 5) share similar characteristics with the OD. Hence the HE or yellowwhite lesion in the retina is frequently detected false positive in the saliency maps of all the models. Then detection becomes conjunctive targets search, (see figure 2 b) which required focused attention. Therefore we need to do some processing on saliency map. The relevance of computational saliency models in detecting Optic disk in retinal images is studied by computing saliency maps using popular model Itti- Koch (IK) [17, 18]. Human visual attention is influenced by both stimulus driven bottom-up influences and goal/task driven top-down influences. In the context of medical images, bottom-up influences correspond to image features whereas top-down influences correspond to the knowledge and expertise of radiologists. 1897

3 The above-mentioned computational model (IK) is bottom-up saliency models and it try to predict saliency solely based on image features. In the present study, we investigated the role of computational model in detection OD in retinal image. Figure 2: Retinal images showing OD as (a) disjunctive and (b) conjunctive target searches The rest of the paper is organized as follows. In section-2, we discuss some related work. In section- 3, we explain the proposed algorithm for OD detection study followed by results in our study. Important results are discussed in section-4 followed by conclusion and future work in section Related Work The OD is considered one of the main features of a retinal image. The process of automatically detecting the OD aims only to correctly detect the centroid of the OD. On the other hand, disc boundary detection aims to correctly segment the OD by detecting the boundary between the retinal and neuroretinal rim. Some methods estimated the boundary of the OD as a circle or an ellipse (e.g., [1], [2], [3], and [4]). Other methods have been proposed for the exact detection of the OD contour (e.g., snakes [5], pyramidal decomposition [6] and a Hausdorff-based template matching [4], gradient vector flow (GVF) [7], which largely has the ability to bridge discontinuities of the edges. [Wisaeng] presented the algorithm that automatically detect OD for low contrast retinal images with a large data set. Since interpreting medical images is highly task dependent, it is generally expected that top-down mechanisms play a very important and significant role in guiding the observers' attention, whereas bottom-up processes might not play an important role. But, a recent study on brain CT images [19] showed that bottom-up mechanisms also play a significant role in guiding the eye movements of neurologists looking for stroke lesions on brain CT images. To the best of our knowledge, there are no other study investigating the role of bottom up saliency in OD detection in retinal images. Based on some psychophysical experiments in the context of searching tumors in chest x-rays, Nodine and Kundel [20, 21] have developed a model of visual search and detection that has three main components: overall pattern recognition (global impression), focal attention to image detail; and decision-making. According to this model, visual search begins with a global response involving the entire retina, in which the context is established and gross deviations from normal are detected. This response initiates a series of checking fixations, using the fovea to resolve ambiguity and fill in detail. Since this Nodine and Kundel's model of visual search in medical images is very similar to that of FIT, we can expect that bottom up saliency models based on FIT would detect salient regions in medical images also. The present study is aimed in investigating whether this is true or not. 3. Proposed Algorithm Since optic disc in retinal images are approximately disjunctive targets, they can be identified through a pre-attentive process. The main dataset is a subset of the STARE Project s dataset [8]. The subset used contains 81 retinal images. We used popular computational bottom-up saliency model IK [7]. IK is a biologically motivated saliency model, which closely follow FIT. This is one of the most popular models for the focused attention stage of visual attention and they predict human fixation points well. IK uses the set of basic image features at multiple scales to compute saliency maps. Steps for proposed algorithm is shown in figure 3. Proposed algorithm Step 1. Pre-processing Typically, there is wide variation in the color, intensity and orientation of retinal image from background as shown in figure 4. The result of attended locations by IK model is shown in figure 4 b. The OD is attended after attending all corners, because there is variation in color, intensity and orientation of retinal image and its background. Hence, pre-processing step is required before using IK [18] model for computing saliency map for OD detection. We used Morphological Opening operator (Φ) applied to the original image. 1898

4 Saliency map is computed using IK[18] model on the images obtained in pre-processing step. Step 3: The Hard Exudates (HE) in a retinal image (see figure 5) share similar characteristics with the OD. Hence the HE or Yellow-White lesion in the Retina is a frequently detected false positive in the saliency maps of all the models. To Suppress the HE we performed binary segmentation. A threshold is applied to saliency map. This thresholding is based on Otsu s algorithm. The objective of thresholding is to convert a gray scale image into a binary image, separating brighter regions from dark background. To correctly detect OD, we need to eliminate HE and yellow-white lesion in the retina as shown in figure 5. For this we used morphologically open operation on binary image, which is than subtracted from thresholded binary image. Figure 3: Steps for proposed algorithm. Figure 5: Sample retinal images showing Hard Exudates and Yellow-White Thing (enclosed in white circles) Figure 4: (a) Original Image and (b) Attended location by IK model. The definitions for opening operation by structuring element B are defined as: OD1= Φ (B1)(C1) where B1 is the morphological structuring element. It is evident that this approach produces a more homogeneous region. The OD1 is subtracted from original image, the resultant image is used to compute saliency map. The result of pre-processing is shown in figure 7 (b). Step 2: Computation of saliency map Step 4: We used morphological opening to remove small objects from an image while preserving the shape and size of larger objects in the image. As opening only eliminate image details smaller than the structuring element used, it is convenient to set the structuring element big enough to cover all possible vascular structures, but still small enough to keep the actual edge of the OD. An example of opening with a disc type structuring element is shown in Figure 7 (d). 4. Result Results are shown in figure 7. The result of pre-processing operation on original image is shown in figure 7 (b). The result of opening operation on thresholded image is shown in figure 7 (c). Detection of OD is shown in figure 7 (d). Table 1 shows the result analysis. The saliency map, which give OD as pop up at first focus of attention FOA, (we have considered it as disjunctive case) proposed algorithm able to detect OD for 43 images out of 53 images. The saliency map, which give OD as pop up at successive focus of 1899

5 attention FOA i.e. which give OD and other region as pop up, (we have considered it as Conjunctive case) proposed algorithm able to detect OD for 14 images out of 28 images. This is shown in figure 6. For a given set of retinal images we found that the proposed algorithm will take approximately same time for both conjunctive and disjunctive type of search. Table 1: Result analysis Time Req. Disjunctive Search Time Req. Conjunctive Search Saliency map with OD as pop-up Saliency map with OD and other region pop-up Pass Fails Pass Fails Discussion and Conclusions In this paper, we have analyzed the relevance of saliency models in detecting abnormalities in two types of medical images. In one study, we analyzed the role of bottom-up saliency in OD detection on STARE Project s dataset [22]. Results obtained indicate that the IK [18] saliency model with mathematical morphology and Otsu s algorithm performs reasonably well in detecting optic disc in retinal images. The results we got is promising and encouraging. Thus, we can say that bottom-up image features, which work well in natural scenes, also play an important role in Medical images. It is generally expected that examining medical images involves top down knowledge to a great degree. However, our studies (with retinal images) indicated that the role for bottom up knowledge is considerably high. Focus of Attention (FOA) Disjunctive Search Focus of Attention (FOA) Conjunctive Search Figure 6: Above two figures shows, the saliency map, for which give OD as pop up at first focus of attention FOA (i.e disjunctive case) and OD as pop up at successive focus of attention FOA (i.e Conjunctive case). Below two figures shows time required by proposed algorithm for both disjunctive and conjunctive cases. (Time required is calculated after saliency map) It is possible that further improvements in performance can be had by using top-down influences such as the role of contralateral symmetry information, influences specific to expertise etc. Likewise, there is a need to investigate if the false positive detection rate reduces. These remain the part of future work. Although the present study is specific to OD detection, we believe that many of these results can be extended to abnormalities with similar characteristics. For instance, many of the results obtained in the study of optic disc detection can be extended to find Hard Exudates and Yellow-White lesion. This saliency map also can be used for other medical images like x-ray, detection of tumors. It is however, not clear to what extent these results can be extended to localized lung lesions such as tumors. However, the present study together with a previous study on stroke CT images [19] show that bottom up saliency plays an important role in medical images. 1900

6 (a) (b) (c) (d) (e) Figure 7: (a) Input Retinal image (b) result after pre processing (c) Saliency map (d) Result after Morphological opening operation (e) OD detected 6. References [1] C. Sinthanayothin, Automated localization of the optic disk, fovea, and retinal blood vessels from digital colour fundus images, British journal of ophthalmology, 1998, [2] L. Gagnon, Procedure to detect anatomical structures in optical fundus images, Processing of Conference on Medical Image, San Diego, 2001, [3] R. A. Abdel-Ghafar, Detection and characterization of the optic disk in glaucoma and diabetic retinopathy, Presented at the Medical Image Understanding Analysis Conference, London, UK, 2004, [4] M. Lalonde, Fast and robust optic disk detection using pyramidal decomposition and Hausdorff-based template matching, IEEE Transection on Medical Image, 2001, [5] H. Li, A model-based approach for automated feature extraction in fundus images, in 9th IEEE International conference Computer Vision, 2003, [6] F. ter Haar, Automatic localization of the optic disc in digital colour images of human retinan, Utrecht University, [7] C. Xu, Gradient vector flow: a new external force for snakes, Proceeding of IEEE conference on computer vision pattern recognition, 1997, 71. [8] L. Itti, C. Koch, and E. Niebur, A Model of Saliency- Based Visual Attention for Rapid Scene Analysis, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp , Nov [9] C. Koch and S. Ullman, Shifts in Selective Visual Attention: Towards the Underlying Neural Circuitry, Human Neurobiology,vol. 4, no. 4, pp , [10] A. Treisman and G. Gelade. A feature-integration theory of attention. Cognitive psychology, 12(1):97{136, [11] L. Elazary and L. Itti. Interesting objects are visually salient. Journal of Vision, 8(3), [12] D. Walther, L. Itti, M. Riesenhuber, T. Poggio, and C. Koch. Attentional selection for object recognition: a gentle way. In Biologically Motivated Computer Vision, pages 251{267. Springer, [13] L. Itti. Automatic foveation for video compression using a neurobiological model of visual attention.image Processing, IEEE Transactions on, 13(10):1304{1318, [14] T. Kadir and M. Brady. Saliency, scale and image description. International Journal of Computer Vision, 45(2):83{105, [15] C. Koch and S. Ullman. Shifts in selective visual attention: towards the underlying neural circuitry. Hum Neurobiol, (4):219{27, [16] J. Wolfe. Asymmetries in visual search: An introduction. Attention, Perception, & Psychophysics, 63(3):381{389, [17] L. Itti and C. Koch. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision research, 40(10-12):1489{1506, [18] L. Itti and C. Koch. Computational modeling of visual attention. Nature reviews neuroscience, 2(3):194{203, [19] H. Matsumoto, Y. Terao, A. Yugeta, H. Fukuda, M. Emoto, T. Furubayashi, T. Okano, R. Hanajima, and Y. Ugawa. Where do neurologists look when viewing brain ct images? an eye-tracking study involving stroke cases. PloS one, 6(12):e28928, [20] C. Nodine and H. Kundel. The cognitive side of visual search in radiology. Eye Movements: From Psychology to Cognition. North Holland, Elsevier Science, pages 572{582, [21] C. Nodine, H. Kundel, et al. Using eye movements to study visual search and to improve tumor detection. Radiographics, 7(6):1241{1250, 1987 [22] STARE Project Website Clemson University, Clemson, SC[online].Available: [23] Siddalingaswamy C, Gopalakrishna rabhu K, Automatic localization and boundary detection of optic disc using implicit active contours, Internation Journal of Computer Applications, vol. 1, no. 7, pp. 1-5, [24] Michael D. Abramoff, Meindert Niemeijer, The automatic detection of the optic disc location in retinal images using optic disc location regression, Conf roc IEEE Eng Med Biol Soc, 1: pp ,