AUTOMATIC FUNDUS IMAGE FIELD DETECTION AND QUALITY ASSESSMENT

Transcription

1 AUTOMATIC FUNDUS IMAGE FIELD DETECTION AND QUALITY ASSESSMENT Gajendra Jung Katuwal 1, John Kerekes 1, Rajeev Ramchandran 3, Christye Sisson 2, and Navalgund Rao 1 1 Chester F. Carlson Center for Imaging Science, Rochester Institute of Technology, Rochester, NY College of Imaging Arts & Sciences, Rochester Institute of Technology, Rochester, NY Flaum Eye Institute, University of Rochester-Medical Center, Rochester, NY ABSTRACT Fundus images are an important diagnostic tool for many retinal diseases. Sometimes the images captured are of low quality and cannot be used for diagnosis requiring repeat image acquisition. So, it is efficient to have an automatic system to assess the quality of the fundus image during the time of image capture. We have developed an automatic approach to assess the quality of the acquired fundus image based upon the inherent symmetry of retinal vessels. We approach the problem of quality assessment in two ways- individual quality assessment of a single fundus image and comprehensive quality assessment of a set of three fundus images of different fields of an eye. Our method also detects the field and side of the fundus image using the position of optic disc and the intensity information in two local windows. Index Terms Fundus Images, Fundus Field, Vessel Density, Quality Assessment 1. INTRODUCTION A retinal fundus image is an image of the inner lining of the eye. It is an important diagnostic tool for many retinal diseases. In tele ophthalmology applications, fundus images are captured in local clinics and are digitally transmitted to another location where an ophthalmologist uses them to perform diagnosis. Sometimes the images captured suffer from different problems like improper positioning, improper illumination, out-of focus, field mis-labeling etc. These problems degrade the quality of a fundus image making them unsuitable for diagnosis. This requires repeat acquisition of images which increases the burden to patients as well as clinics. So, it is desirable to have an automatic system to assess the quality of the images during image acquisition. Several approaches to automatically determine the quality of retinal images have been developed. Lee and Wang [1] calculated the quality score of the fundus image by crosscorrelation of the image intensity histogram with an ideal The authors gratefully acknowledge the support of this work through a summer micro grant from the RIT Chester F. Carlson Center for Imaging Science. template intensity histogram derived from the highest quality images. Lalonde et al. [2] used two different features: the local distribution of the pixel intensities and the distribution of edge magnitudes. Usher et al. [3] used area of segmented vessels as a measure of the quality. Fleming et al. [4] were the first authors to approach quality assessment as a multi-class classification problem and their quality score was divided into two aspects: image clarity and field definition. Niemeijer et al. [5] have used a compact representation of image structures with raw histograms of the R, G, and B color planes to train their classifiers. They use Image Structure Clustering to provide a compact representation of the structures found in an image. Hunter et al. [6] used three aspects of macula vesselsdistance from fovea, contrast and quantity to calculate the quality measure. Giancardo et al. [7] have used local vessel densities (LVDs) and color information as the features in their method. We have developed an automatic system for field detection and quality assessment of the fundus images. To the best of our knowledge, this is the first work in field and side detection. Our field detection method uses the position of the optic disc and the sum of intensities of two local windows in the retinal image. Our quality assessment method is distinct from others since it assesses the quality of all the fields of the retina unlike other method which focus only on macula centered field. We also calculate a global quality score for a set of retinal images of an eye unlike other methods which do not utilize the overlapping area between the fields. For quality assessment, the training images have been classified in quality from 1 to 5 and classifier(s) designed to give a quality metric on a 1-5 scale. Two approaches for quality assessment have been taken. First, quality assessment of individual images are done using global vessel density (GVD) (total vessel area in vessel segmented fundus image) and comparisons of LVDs (to check the inherent symmetry of vascular structure). Second, comprehensive quality assessment of three overlapping fields of an eye is performed. GVD of respective quadrant and Kullback Leibler Divergence (KLD) are used for individual quality assessment of each quadrant of three field images. All individual quality levels of the quadrants are then com-

2 (a) Surf plot of normalized crosscorrelation matrix between optic disc (b) Slope of 100 largest correlation coefficients and area of peak as a measure of the template and quality 5 OD1 image. Red narrowness of the peak. peak signifies the location of optic disc. Fig. 1: Field Positioning: There are three fields for a fundus image - optic disc centered (field 1), macula centered (field 2) and macula to temporal (field 3). In addition to fundus images, an external image of each eye is captured. Thus, in total, 8 images of each patient are taken. Fig. 2: Demonstration of true hit: detection of optic disc in field 1. bined to give a single quality metric. 2. FIELD AND SIDE DETECTION Dataset Our images were captured by Zeiss Visucam NM Pro Fundus Camera and our method takes three fields for each eye (see Figure 1 ). Our dataset (see Table 1) has 88 images taken from 13 patients. There are four different fields (including external), two sides and five quality levels making = 40 different variants. Table 1: 88 non-mydriatic images were taken from 13 patients. Quality of the fundus images were graded on a 1-5 scale jointly by a CRA and an ophthalmologist. Class/Quality No. of Images Total No. of Images 88 The optic disc is more distinct than macula in fundus image and hence more easier to detect. So, the algorithm focuses on the detection and location of the optic disc. Normalized cross-correlation with a template extracted from the highest quality image is done to find its position. Since this method is highly susceptible to false detection, a check for false detection is vital. False hits are detected using narrowness and height of the correlation peak. Figure 2 shows detection of optic disc in field 1 image which is a true hit. The slope of the peak is , area of the peak is and the height of peak or highest correlation coefficient is Generally, in excellent images the peak is greater than 0.7, slope is greater than 0.6 and the area (a) Surf plot of normalized correlation matrix of correlation between OD3 and optic detection, area of the peak is really high (b) Since there is no optic disc and it is false disc. There is no distinct peak and, slope and height of the peak are small. Fig. 3: Demonstration of false hit: detection of optic disc in field 3. is smaller than Figure 3 shows detection of optic disc in field 3 image which is a false hit. There is no distinct peak in Figure 3a and area of the peak is really high while slope and height of the peak are small(see Fig 3b) Detection of OD2, OS2, OS1/OD1 and OD3/OS3 optic disc is at left, center and right in OS2, OD2 and OD1/OS1 fields respectively, and is absent in OD3/OS3 fields. For OD1/OS1 and OD3/OS3 further check is necessary to distinguish the side Distinction of side: Field 3- OD3/OS3 and Field 1- OD1/OS1 Detection of the macula in a fundus image with a low false positive rate is a difficult task. However, the region with the macula is darker than other region. This information is used to distinguish the side of field 3 and field 1 images. Sum of the intensities in left and right local windows S1 and S2 are calculated. Since the macula is at right side of the OD2 and is mirrored for the left side, following decision rule is applied: S1 OD2 OS2 S2 (1) Macula is at the left in field 1 image. So, OD1 and OS1 are distinguished by the decision rule opposite of Equation 1.

3 3. INDIVIDUAL QUALITY ASSESSMENT Individual quality score of a fundus image is primarily dependent on the symmetry and the amount of segmented vascular structure Vessel Segmentation Our vessel segmentation method is inspired by the work of [8] with some modification to suit our quality assessment algorithm. Since retinal vessels are red, the green channel provides the maximum contrast against the background. So, Stationary Wavelet Transform (SWT) [9] is applied on the green channel of RGB fundus image to extract the vascular structure. Three wavelet levels (1-3) are summed and lowest 15% of the coefficients are taken as vessels. There is a lot of salt and pepper noise in the binary image after thresholding which is minimized by median filtering (see Figure 4b). Then, small connected objects with pixel area less than 0.5% of the total pixels of the image are removed to get the image as shown in Figure 4c. After that, small holes are filled by dilation as shown in Figure 4d. The dilated vessels are then thinned to their center lines. This step equalizes the importance of the vessels and increases the reliability of features related with vessels. Finally, a circular mask is applied to get the vascular structure as shown in Figure 4e. Vascular structure in a fundus image is naturally top-down symmetric. In low quality images, vessels of degraded part cannot be extracted and hence the extracted vascular structure becomes asymmetric. This is an important feature of our algorithm. To check symmetry, the vessel image is divided into local windows as shown in Figure 4f and the vessel density (number of non-zero pixels) is calculated in each window. Thus, following features are extracted from vascular structure: Global Vessel Density(GVD) Local Vessel Densities(LVDs) Difference between LVDs of top and bottom local windows. Difference between sum of LVDs in top half and bottom half. These features are properly normalized and used to train a multi-class SVM with linear kernels. Since we had a very small dataset, for preliminary tests, the whole dataset was used to train k-nn classifier with 10 fold cross validation. Its result was used to get an initial estimate and choose features. Finally, 5 binary SVMs with linear kernels were combined in One Vs. All scheme to do multi-class classification. Among 88 fundus images, 12 external images are disregarded and the remaining 76 are used for classification. Out of 76 images, 5 images of qualities 1 to 5 for each field are taken for test set. So, all together, test set has 15 images and training set has 61 images. (a) Binary image after thresholding by taking lowest 20 % wavelet coefficients (b) Median Filtering is done to remove noise (c) Small objects removed to clean the image (d) Dilated to fill the holes (e) Vessels are thinned to their center lines (f) Dividing the image in to number of windows and calculating LVDs for symmetry and a circular mask is applied. check. Fig. 4: (a)-(e) Vessel Segmentation. (f) Vessel Symmetry Check 4. COMPREHENSIVE QUALITY ASSESSMENT This method predicts the overall quality level of a set of field images of any eye utilizing the overlap between them. As shown in Figure 5, each field image is divided into 4 quadrants in clockwise order. Roughly two quadrants are common for neighboring fields. Each quadrant is treated as a separate entity during classification and was assigned a separate quality level, jointly by a CRA and an ophthalmologist. For each quadrant, the following features were extracted: GVD- number of vessel pixels in the quadrant. KLD of the normalized histogram of Local Binary Pattern of the quadrant from that of corresponding quality 5 quadrant. A multi-class SVM with rbf kernel was trained to predict the quality levels of the quadrants. Maximum of the quality levels of overlapping quadrants was taken. Finally, quality levels of all the quadrants were averaged to give a final quality

4 Table 3: Results for Individual Quality Assessment Predicted Class True Class (a) Confusion matrix for individual quality assessment using multi-class SVM with linear kernel. Fig. 5: Overlapping Fields. Each field image is divided into 4 quadrants in clockwise order. Roughly two quadrants are common for neighboring fields. Each quadrant is treated as a separate entity during classification. Even if a part of an image is degraded and the corresponding part in another overlapping image is good, the information of the fundus can be obtained. Class Micro-averaged F1-measure F1-measure (b) Classification result for individual quality assessment using multi-class SVM. F1- measure which is overlap between true and estimated classes is much higher for quality 1 and 2 images since they are more distinguished and are easier to identify. Multiple Binary SVMs were combined in One Vs. All to do multi-class classification. Table 2: Results for Field and Side Detection Predicted Field True Field (a) Confusion matrix for field detection. Most of the error is due to misclassification of other fields to field 3. Predicted Side True Side (b) Confusion matrix for side detection. Most of the misclassifications are for quality 1 images. level for the set of 3 field images. Since the true comprehensive quality scores for the set of three field images were not available, result analysis has not been done. However, from visual inspection, our algorithm seems to work pretty well Field Detection 5. RESULTS For field detection, F-measure of 0.86 was achieved. In Table 2a, it can be noticed that major portion of the error is due to the misclassification of other fields to field 3. This kind of error occurs for very low quality images (quality 1 & 2). Due to the degradation of the image, optic disc cannot be detected and consequently it is assigned as field 3 as per our assumption Individual Quality Assessment Table 3a shows the confusion matrix for the multi-class SVM with linear kernel. The results in F1-measure are in Table 3b. F1-measure which is overlap between true and estimated classes is much higher for quality 1 and 2 images since they are more distinguished and are easier to identify. Micro average F1-measure is This result is justified since there is lot of similarity between neighboring class images (For example class 3 and class 4) and high chance of misclassification among them. In Table 3a, it can be noticed that maximum misclassification distance is two classes. For example, a class 5 image is misclassified farthest to class CONCLUSION The field and side detection algorithm shows promising result with failures only for very low quality images which basically do not contain any retinal structures and are supposed to be rejected by our system. So, misclassification for very low quality images can be ignored. To remove this error, a preliminary quality detection can be done using only GVD to give a rough quality score. GVD alone is enough to detect the very low quality images and there is no need of field/side detection if the GVD is very low. The result of the individual quality assessment method seems reasonable. Class 1 and Class 2 have higher F1-measure as very low quality images do not contain enough retinal structures and hence are distinct. Thus, the problem of field and side detection has been addressed with failures only in very low quality images. Quality assessment can be improved by modifying the current features and adding color information. Accuracy of field/side detection can be increased up to 100 % by filtering out very low quality images using GVD, as suggested above. Since, accuracy of the method is more important than the speed in this problem, all processing is done on original image without sub-sampling. Sub-sampling can be done to decrease computation load with out compromising the accuracy. This multiclass classification system can be converted to binary classification to give pass/fail result by thresholding at class REFERENCES [1] S Lee and Yiming Wang, Automatic retinal image quality assessment and enhancement, Proceedings of SPIE

5 Image Processing, vol. 3661, no. February, pp , [2] Lalonde, Gagnon, and Boucher, Automatic visual quality assessment in optical fundus images, Proceedings of Vision Interface, pp , [3] Usher, Himaga, and Dumskyj, Automated assessment of digital fundus image quality using detected vessel area, Proceedings of Medical Image Understanding and Analysis, pp , [4] Alan D Fleming, Sam Philip, Keith A Goatman, John A Olson, and Peter F Sharp, Automated assessment of diabetic retinal image quality based on clarity and field definition., Investigative ophthalmology & visual science, vol. 47, no. 3, pp , Mar [5] Meindert Niemeijer, Michael D Abràmoff, and Bram van Ginneken, Image structure clustering for image quality verification of color retina images in diabetic retinopathy screening., Medical image analysis, vol. 10, no. 6, pp , Dec [6] Andrew Hunter, James a Lowell, Maged Habib, Bob Ryder, Ansu Basu, and David Steel, An automated retinal image quality grading algorithm., in Conference proceedings :... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference, Jan. 2011, vol. 2011, pp [7] L Giancardo, F Meriaudeau, T Karnowski, E Chaum, and K Tobin, Quality assessment of retinal fundus images using elvd, New developments in biomedical engineering, pp , [8] Peter Bankhead, C Norman Scholfield, J Graham McGeown, and Tim M Curtis, Fast retinal vessel detection and measurement using wavelets and edge location refinement., PloS one, vol. 7, no. 3, pp. e32435, Jan [9] G. P. Nason and B. W. Silverman, The stationary wavelet transform and some statistical applications, 1995, pp , Springer-Verlag.