1.

Introduction

A majority of the people suffering from diabetes mellitus will eventually develop diabetic retinopathy (DR). At the final stage of DR, sufferers may lose their eyesight. DR is one of the leading causes of blindness and can be controlled if detected early. But for population-based assessment, the task of grading each retinal fundus image is both time consuming and labor intensive. With the advent of digital fundus photograph technology and the availability of fast computers, systems are being designed to detect DR automatically. Microaneurysms (MAs), which are basically the saccular enlargement of the venous ends of retinal capillaries, are said to be the first sign of DR. MAs appear and disappear during the early course of retinopathy.^1,2 The MA count and turnover in digital color fundus images are important measures of DR progression.^3,4 Therefore, accurately detecting MA is not only important for DR detection, but also it may assist in monitoring DR progression.

The MAs have diameters between 10 and $100\mu \mathrm{m}$, are round in shape, and their color is similar to blood vessels (BVs) (red),^5,6 as shown in Fig. 1. Detection of MAs is challenging due to the variation in MA size, low and varying contrast, uneven illumination, and variation in fundus image background. MA detection is not a new topic; many researchers have worked on it since the early 1980s.^7,8 Though the performance of the automatic system in digitized fluorescein angiograms is somewhat on par with a human grader,⁹ it is considered to be an invasive method, as fluorescein sodium dye is injected into the eye. The associated risk of complication or adverse reaction can include transient nausea, occasional vomiting, and so on, and in very rare cases can cause death. Thus most of the current research is moving toward color fundus photography, which is a noninvasive imaging method. In the absence of contrast enhancing agent, color fundus images inevitably suffer from low contrast. The performance of the fundus image-based system is, as expected, limited and remains as an open issue in retinal image analysis.

Fig. 1

Example of MA, where row 1 shows the MA in (a) full-color RGB and (b) a close-up view; row 2 is (c) in green band and (d) with a close-up view.

Some of the well-known approaches used in MA detection include template matching in wavelet domain, scale-adapted blob analysis with semisupervised learning scheme, ensemble-based system, double-ring filter, local rotating cross-section profile analysis, multiscale correlation coefficients, and pixel classification technique.^10–16 In this work, we propose to use curvelet transform (CT) for MA detection as it can detect curve singularities. Our preliminary work indicated promising results (Shah et al.¹⁷ IOVS 2015;56: ARVO E-Abstract 5266).

2.

Materials and Methods

2.2.

Automated Microaneurysm Detection System

Figure 2 depicts the complete procedure of the proposed MA detection system which consists of the following steps:

• 1. Candidate selection

  • a. BV extraction from green band of raw image, preprocessing of green band, removal of BV from preprocessed green band, and MA candidate selection using local thresholding.

  • b. Background estimation using gray band of raw image.

  • c. Removal of candidates from step a that appear in the background.

• 2. Candidate features extraction

  • a. Color-based features.

  • b. Hessian-based features.

  • c. Curvelet-based features.

• 3. Candidate classification

  Since the dataset consists of different sizes of images, the height of all the images was resized to a standard 800-pixel while maintaining their original width/height ratio. After resizing, each color fundus image was divided into red band ($G_{\mathrm{r}}$), green band ($G_{\mathrm{n}}$), and gray band ($G_{\mathrm{y}}$). We preprocessed the $G_{\mathrm{n}}$ by shade correction, followed by histogram equalization to obtain the preprocessed image denoted by $G_{\text{npr}}$. The BVs were extracted from the $G_{\mathrm{n}}$, using two-dimensional (2-D) Gabor wavelet,²⁰ and removed from the $G_{\text{npr}}$ to obtain BV-removed image denoted by $G_{-\mathrm{BV}}$. The MA candidates were obtained in two parallel steps, namely local thresholding and statistical features-based technique. The MAs have low intensity in green band. Using local thresholding,²¹ MA candidate pixels with low intensities could be selected from $G_{-\mathrm{BV}}$ as potential candidates. Also from the gray band image ($G_{\mathrm{y}}$), based on pixel contrast and standard deviation, we could identify MA candidates. Only those candidates which were found by both of these methods were retained. In both MA candidate selections, thresholds were kept at low value to maximize the detection of MAs. Hence, many true MAs were detected along with hundreds of false positives (FPs). These FPs were mainly from the traces of the BVs, the background and bits of larger red lesions.

Fig. 2

Automated MAs detection system flowchart.

2.3.

Candidate Feature Extraction

In general, MAs are red in color and have round shape. Based on their morphology and intensity, we used three feature sets to describe the MAs, namely color-based, Hessian matrix-based, and curvelet coefficients-based features. The color features include (a) standard deviation and mean intensity values in red and green bands, and (b) the histogram of S and V bands in HSV color space. The BVs and other linear objects can be detected based on the eigenvalues of Hessian matrix.²² The features we used based on Hessian matrix are thus the eigenvalues, their product, and their ratio.

The basic concept of curvelets²³ is to represent a curve as a superposition of multiple functions of various lengths and widths obeying the scaling law $\text{width}\approx {\text{length}}^2$. CT has a highly redundant dictionary, which can provide sparse representation of signals that have edges along regular curve. It is localized in angular orientation in addition to localization in spatial and frequency domains—a very important feature missing in the classic wavelet transform. Initial construction of curvelets has been redesigned and was reintroduced as fast digital CT.²⁴

Curvelets are used in many medical image analysis applications like computed tomography,²⁵ breast cancer diagnosis in digital mammogram,²⁶ ulcer detection,²⁷ retinal image analysis,²⁸ and so on. Most natural images/signals exhibit line-like edges, i.e., discontinuities across curves (so-called line or curve singularities). Traditional wavelets perform well only at representing point singularities, since they ignore the geometric properties of structures and do not exploit the regularity of edges.²⁹ The solution to this problem and some other limitations of the wavelet was provided by CT.²³ Unlike the isotropic elements of wavelets, the needle-shaped elements of this transform possess very high directional sensitivity and anisotropy.²⁹ The algorithm of CT is shown in Fig. 3. It is based on two windows, namely scale window $V$ and radial window $W$, and consists of four steps: (a) compute the 2-D Fourier transform of the original image, and (b) for each scale $s$ and orientation $n$, estimate frequency window $U_{s,n}$ as a product of the scale and radial windows, (c) wrap this product around the origin, and (d) compute a 2-D inverse fast Fourier transform to derive the curvelet coefficients. More details can be found in Candes’ paper.²⁴ In digital implementation of the CT, the two main parameters are

• a. number of scales and

• b. number of orientations

at the coarsest level. We found in our case that 2 scales and 16 orientations work well. Based on curvelet coefficients, we calculated aspect ratio, circulatory, mean energy, and standard deviation of energy.

Fig. 3

Steps of CT.²⁸

3.

Results and Discussion

Out of the 50 images, only 37 images contain MAs, while the remaining 13 images do not contain any MA. The total number of MAs in these 37 images is 336. The results of the proposed system and those previously reported in literature are shown in Table 2. Out of the 336 MAs, the proposed approach was able to detect 162 MAs, achieving a sensitivity of 48.21% with 65 FPs per image (FPPI). This result is favorably comparable with the state-of-the-arts, although only simple rule-based classifier was implemented. Unlike the method by Adal et al. that employed four supervised classifiers and 87 features in total to select an optimum classifier–feature pairing to remove FPs, the computation of the proposed system is simpler and faster albeit at the expense of higher FPPI. In this work, we aimed to achieve high sensitivity in MA detection, i.e., to detect as many MAs as possible from fundus images. We used local thresholding technique in identifying MA candidates and kept the threshold to a low value as our approach to achieve maximum possible sensitivity, at the cost of hundreds of FPPI. These FPs at this initial stage were mainly due to background of the fundus images and BVs. We used statistical features to estimate the background and remove those candidates that are from background. In addition, during the BV extraction stage, we kept the threshold to a low value so that only true BVs are extracted. This helps us in detecting MAs near the BV and ultimately improving the sensitivity. However, this resulted in introducing many FPs as there were many traces of BVs. To eliminate those BVs, we used the Hessian matrix-based features. Thus, the proposed system can detect MA near the BVs with reasonably good specificity. Figure 4 illustrates the examples of MA detected in close vicinity to BVs.

Table 2

Result comparison of different MA detection methods.

Fig. 4

Examples of MA that are very close to the BVs but detected by the proposed system: (a)–(d) full-color images and (e)–(h) in green band.

As previously mentioned, our objectives are different, in that we have targeted for better sensitivity whereas they focused on specificity. As the “good” MAs (clear circular dark red spots) form only a fraction of all the MAs in this dataset, this has a significant impact on how robust one approach can be optimized in coping with the variance of MA features. Using larger number of features and more complex classifiers, Adal et al. were successful in achieving very good specificity. Increasing its sensitivity (i.e., accepting MAs with larger variance) may however not be straightforward.

Figure 5 shows the free-response receiver operating characteristic (FROC) curve of the proposed system. Based on local thresholding alone, the proposed system would have hundreds of FPPI. More than 42% of those FPs could however be removed using color features, with only 2.3% of the MAs being lost. We started with 178 MAs detected out of possible 336 (with 250 FPPI), after local thresholding. The results were improved to 170 MAs with 144 FPPI, after MA candidate selection based on color features. However, to achieve very high specificity ($<100\mathrm{FPPI}$), additional features are required to differentiate the actual MAs and the FPs. In this case, we optimized the results using Hessian and curvelet features simultaneously.

Fig. 5

FROC plot of the proposed system.

To our best knowledge, CT is used for the first time in MA detection. CT is fast and robust at detecting objects with curved singularities. We used features including shape parameters based on curvelet coefficients to discriminate between MAs and non-MAs. The results indicate that curvelets are very effective at detecting round objects such as MAs. Table 2 further shows our approach achieves the higher sensitivity than other reported approaches. Admittedly, the FPPI is higher than Adal’s but a simpler solution is perhaps better, as the Occam’s razor principle suggests.

The ROC dataset is a very challenging dataset. It has been observed that general image quality, noise, and low and varying contrast make it difficult to detect MAs in this dataset. Figure 6 depicts some of the MA candidates detected by the proposed system. The FPs are mostly from BVs and hemorrhages (large red lesions). Some of the images are very dark, hence in those images the background has also contributed to the FPs. Factors such as variation in fundus image background, low and varying contrast, and artifact are found to further limit the MA detection rate. The cases where the proposed system failed to detect MAs are shown in Fig. 7 and can be summarized into three categories:

Fig. 6

Objects encircled in black are true positives, encircled in white are false negatives, whereas objects inside the white squares are FPs: (a) full-color image while (b) in green band.

Fig. 7

Different cases where the proposed system was unable to detect the MAs: (a)–(d) examples of missed MA because the center pixel does not have the minimum intensity, (e)–(h) examples of missed MA because their colors are very faint, and (i)–(l) examples of missed MA due to abnormal shape.

One possible approach to improve the sensitivity and specificity of the proposed MA detection system is the utilization of multispectral imaging (MSI) approach. Applications of MSI have shown promising results in different areas of biomedical image analysis ranging from human forearm imaging to skin chromophore mapping,^34–38 with several applied to retinal image analysis.^39–42 A recent study on retinal vein occlusion demonstrated that MSI was able to define vascular abnormalities at a comparable performance as fundus photography, fundus fluorescein angiography, and optical coherence tomography.⁴³ In MSI, image data are captured at specific nonoverlapping frequency bands. Thus, certain features within the field of view can be highlighted. If applied in MA detection, we expect better BV extraction and background separation may be achieved. These will result in less stringent requirement at the classifier stage, and hence improvement in sensitivity.

4.

Conclusion

We have explored a new technique for MA detection. We observed that the main sources/contributors of FPs in automated MA detection such as proposed one are image background and BVs while hemorrhages are the third category of FPs although with fewer in numbers. The proposed system has a high sensitivity and is able to detect MAs near the BVs. Our future work includes investigation of a means to detect hemorrhages and fine BVs to further improve the specificity of our proposed MA detection system.