2.1.

Retinal Image Clarity Assessment Review

Retinal image clarity assessment techniques can be generally divided into spatial and transform-based depending on the domain from which the quality features are calculated. Although spatial RIQA techniques were more widely adopted in literature, recently several transform-based algorithms have been introduced and shown to be highly effective in RIQA.

2.2.

Content Literature Review

Several RIQA works considered retinal field definition and nonretinal image differentiation alongside clarity issues. For field definition evaluation, Fleming et al.²³ presented a two-step algorithm in which the vessel arcades, OD, and the fovea were initially detected. Then, several distances related to these structures were measured to assess the image’s field of view (FOV). Other algorithms relied only on validating the presence of the OD,⁴³ fovea,⁹ or both³⁰ in their expected locations. As for outlier detection, Giancardo et al.¹⁹ and Sevik et al.³⁰ used RGB information for the identification of outliers. Yin et al.³¹ used a two-stage approach to detect outlier images in which a bag of words approach first identified nonretinal images. Next, these nonretinal images were rechecked by comparing them to a set of retinal images using the structural similarity index measure.⁴⁵

In this work, a no-reference RIQA algorithm intended for automated DR screening systems is proposed. The introduced algorithm evaluates five clarity and content features: sharpness, illumination, homogeneity, field definition, and outliers. Sharpness and illumination features are calculated from the UWT detail and approximation subbands, respectively. For homogeneity assessment, wavelet features are utilized in addition to features from the retinal saturation channel that is specifically designed for retinal images. Field definition and outlier evaluation are performed using the proposed sharpness/illumination features and color information, respectively. All proposed feature sets are separately validated using several datasets. Then, all the features are combined into a larger quality feature vector and used to classify a dataset having various quality degradation issues into good and bad quality images. Finally, the overall performance of the introduced RIQA algorithm is thoroughly analyzed and compared to other algorithms from literature.

4.1.

Sharpness

Medically suitable retinal images must include sharp retinal structures to facilitate segmentation in automatic systems. Generally, sharp structures are equivalent to high-frequency image components. Wavelet decomposition can separate an image’s high-frequency information into its detail subbands. Thus, sharp retinal images would have more information content within their wavelet detail subbands corresponding to larger absolute wavelet coefficients compared to blurred images. Figure 2 shows the level 3 detail subbands of sharp and blurred retinal images. More retinal structure-related information appears in the different detail subbands of the sharp image in comparison to the amount of information within the blurred image’s detail subbands.

Fig. 2

Level 3 green channel detail subbands of sharp (first row) and blurred (second row) retinal images from DRIMDB.³⁰ From left to right: color image, horizontal, vertical, diagonal subbands.

In the previous work by AbdelHamid et al.,⁴⁴ retinal quality assessment was based on the intuition that good quality images have sharp retinal blood vessels. Extensive analysis was made to find the single wavelet level having the most relevant blood vessel information. Sharpness quality evaluation was then performed using features calculated solely from that single wavelet level. However, the analysis demonstrated the dependence of the relevant wavelet level on the amount of blurring within the image.

In this work, a more generic approach for retinal image sharpness assessment is adopted. Retinal images are decomposed into five wavelet levels (L1, L2, L3, L4, and L5) and all levels are involved in the computation of the sharpness features. Nirmala et al.⁴¹ have shown that there is a relation between retinal structures and different wavelet levels. They showed that for $512\times 512\text{pixel}$ images, blood vessel and OD information was more relevant in the detail subbands of levels 2 to 4, whereas information related to the macular region was more significant in levels 1 to 3. Hence, considering L1 to L4 for sharpness features computation has the advantage of taking into account sharpness information of all the different retinal structures. Level 5 has also been used for sharpness feature calculation to investigate its relevance to retinal image sharpness assessment. Another modification to our previous work is the use of UWT, which preserves more image information than the DecWT due to the absence of the decimation step.⁴⁸

Sharpness RIQA algorithms commonly use only the green channel for feature computation because of its high contrast.^{7,17–19,28–30,43,49} However, observation of retinal images has shown that retinal structures in the red channel of sharp images were generally clearer than in those of blurred images. In this work, the proposed sharpness feature set includes four wavelet-based measures computed from both the red and green detail subbands. The wavelet Shannon entropy,⁵⁰ mean, and interquartile range (IQR) were used to measure the information content, expectation, and dispersion of the detail subband coefficients, respectively. The three sharpness features are given by the following equations:

The fourth sharpness measure used is the difference in wavelet entropies between the test image and its blurred version (${\text{WEntropy}}_{\mathrm{diff}}$). Sharp images are generally more affected by blurring than unsharp images. ${\text{WEntropy}}_{\mathrm{diff}}$ separates sharp from unsharp images by measuring the effect of blurring on the test images. Gaussian filters are commonly used in RIQA literature^9,15,26 as a degradation function to produce blurring effect. Several Gaussian filter sizes with different standard deviations were tested to compare their effect on retinal image sharpness starting by a $3\times 3$ Gaussian filter then incrementally increasing the filter size. In our final experiments, a Gaussian filter of size $15\times 15$ and standard deviation 7 was used to get the blurred version of the test image as it achieved noticeable blurring within the retinal images affecting both the thin and thick blood vessels. Smaller filters were found to scarcely cause any visually detectable blurring within the images.

The sharpness feature set includes the wavelet-based entropy, mean, IQR, and WEntropy_diff. All features were calculated from levels 1 to 5 of both the red and green detail subbands. Further feature selection will be performed in Sec. 5.1 based on performance analysis of the sharpness feature set on two different datasets.

4.2.

Illumination

An increase or decrease in the overall image illumination could affect the visibility of retinal structures and some disease lesions. The contrast of the image is also decreased, which could negatively affect the suitability of the image for medical diagnosis. It has been shown by Foracchia et al.⁵¹ that illumination in retinal images has low spectral frequency. Low-frequency information of the multiresolution decomposed images can be found in their approximation subbands. Most image details are separated in the detail subbands of higher wavelet levels. Hence, the L4 approximation subband is almost void of sharpness information consisting mainly of illumination information. In this paper, the adequacy of using L4 RGB wavelet-based features for retinal illumination assessment is explored.

Figure 3 shows examples of the RGB channels of well-illuminated, over-illuminated, and under-illuminated retinal images. The overall image illumination is seen to be directly reflected in the exposure of each of the RGB channels. RGB color information has been used before for retinal image illumination evaluation by several researchers. Niemeijer et al.¹⁸ created a feature set including information from five histogram bins for each of the RGB channels. Yu et al.⁷ calculated seven statistical features (mean, variance, skewness, kurtosis, and quartiles of cumulative density function) from each of the RGB channels. The illumination feature set used includes the mean and variance L4 approximation subband of the RGB channels.

Fig. 3

Examples of retinal images of varying illumination from DR2:⁹ well-illuminated (first row), over-illuminated (second row), and under-illuminated (third row). From left to right: color image, red channel, green channel, blue channel.

4.3.

Homogeneity

The complexity of the retinal imaging setup along with the naturally concave structure of the retina can result in nonuniformities in the illumination of retinal images. As a result, the sharpness and visibility of the retinal structures can become uneven across the image. This in turn would affect the reliability of the image for medical diagnosis. Retinal image homogeneity is commonly addressed in the literature using textural features^15,17,28,30 that are not adapted for retinal images and can be computationally expensive.²⁷

In this section, homogeneity measures specifically designed for retinal images are introduced. Retinal images from DR2, DRIMDB, and MESSIDOR datasets were inspected and analyzed in the process of the homogeneity features development. All images used for the development of the homogeneity features were excluded from any further homogeneity tests to avoid classification bias.

4.3.1.

Retinal saturation channel homogeneity features

Basically, the colors observed in retinal images depend on the reflection of light from the different eye structures. The radiation transport model (RTM)^52,53 shows that light transmission and reflection within the eye mainly depend on the concentration of hemoglobin and melanin pigments within its structures.⁵⁴ Red light is scarcely absorbed by either hemoglobin or melanin and is reflected by layers beyond the retina. This results in the usually over-illuminated, low-contrast red channel as well as the reddish appearance of the retinal images. Blue light is strongly absorbed by hemoglobin and melanin as well as by the eye lens.⁵⁵ This results in the commonly dark blue channel of retinal images. Green light is also absorbed by both pigments but less than the blue light. Specifically retinal structures containing hemoglobin, such as blood vessels, absorb more green light than the surrounding tissues resulting in the high vessel contrast of the green channel.

The RTM directly explains the theory behind the well-known notion of retinal images having overexposed red channels, high contrast green channels, and dark blue channels. Based on this theory, we propose a saturation channel specifically suitable for retinal images. The proposed saturation channel was inspired by the saturation channel of the HSV model ($S_{\mathrm{HSV}}$) and is referred to as the retinal saturation channel ($S_{\text{retina}}$). By definition, saturation is the colorfulness of a color relative to its own brightness.⁵⁶ In the HSV color model, saturation is calculated as the most dominant of the RGB channels with respect to the least prevailing color as given by the following equation:

Eq. (4)

$$S_{\mathrm{HSV}}=\frac{\text{maximum}(\mathrm{R},\mathrm{G},\mathrm{B})-\text{minimum}(\mathrm{R},\mathrm{G},\mathrm{B})}{\text{maximum}(\mathrm{R},\mathrm{G},\mathrm{B})}.$$

The $S_{\text{retina}}$ considers the findings of the RTM in creating a saturation channel better suited for retinal images. Essentially, the $S_{\text{retina}}$ has two main modifications over $S_{\mathrm{HSV}}$:

• 1. Based on the RTM, red is the predominant color in retinal images. Thus, the proposed saturation channel is calculated as the colorfulness of the red channel.

• 2. In order to more precisely capture retinal image illumination variations, the proposed retinal image saturation channel is calculated as the colorfulness of red with respect to both the green and blue channels. These are accounted for in the $S_{\text{retina}}$ equation as $(R-G)/R$ and $(R-B)/R$, respectively.

The equation of the newly proposed $S_{\text{retina}}$ is then given by

Eq. (5)

$$S_{\text{retina}}=\{\begin{array}{ll}\frac{(R-G)}{R}*\frac{(R-B)}{R}& \text{for}R\ne 0\\ 0& \text{for}R=0\end{array},$$

where the R, G, B denote the red, green, and blue components of the retinal image, respectively.

Figure 4 shows examples of retinal images of various homogeneities along with their $S_{\text{retina}}$ channel and $S_{\text{retina}}$ histogram. Evenly illuminated retinal images as in Fig. 4 (rows 1 and 2) resulted in Gaussian-like shaped $S_{\text{retina}}$ distributions centered within the middle region of the histogram. On the other hand, for nonhomogeneous retinal images as in Fig. 4 (rows 3 and 4), the distribution shape of the $S_{\text{retina}}$ was altered as well as being shifted leftward. The difference in histogram positioning between the homogenous and nonhomogenous images is accounted for by measuring the percentage of pixels in middle range of the $S_{\text{retina}}$ ($P_{\mathrm{mid}}$). A seventy-five percent middle intensity range was considered to take into consideration different lightening conditions of homogeneous retinal images. Furthermore, several $S_{\text{retina}}$ features were used to evaluate the images’ homogeneity due to depicted differences between the $S_{\text{retina}}$ histogram distributions of homogenous and nonhomogenous retinal images.

Fig. 4

Examples of homogeneous retinal images from (row 1) MESSIDOR,⁴⁶ (rows 2) DR2,⁹ and (rows 3 and 4) nonhomogeneous retinal images from DR2.⁹ From left to right: color image, $S_{\text{retina}}$ channel, and $S_{\text{retina}}$ channel histogram.

The homogeneity feature set includes the $S_{\text{retina}}$ mean, variance, quartiles, IQR, skewness, kurtosis, coefficient of variance (CV), $P_{\mathrm{mid}}$, energy, and entropy. More features will be added to the homogeneity feature set in the next section.

4.3.2.

Wavelet-based homogeneity features

Wavelet-based features are introduced in this section to complement the features derived from the $S_{\text{retina}}$ channel to comprehensively address retinal image homogeneity.

• i. Difference between wavelet entropies of the red and green channels

  As explained by the RTM theory summarized in Sec. 4.3.1, the green channel of retinal images typically has higher contrast than the oversaturated red channel. Higher green channel contrast is in turn equivalent to higher information content which can be measured by the wavelet entropy of the detail subbands. So for well-illuminated retinal images, wavelet entropies of the green channel’s detail subbands are commonly larger than those of the red channel’s detail subbands. However, illumination variations in retinal images are more prominent in the red than the green channels as shown in Fig. 5. Therefore, nonhomogenous retinal images tend to show larger red channel wavelet entropies than green channel wavelet entropies. Figure 6 shows examples of homogeneous and nonhomogeneous images along with their green and red L3 wavelet entropies. The difference between L3 red and green channel wavelet entropies is added to the retinal image homogeneity feature set.

• ii. WaveletEntropyL3/ WaveletEntropyL1 of green channel detail subbands

  Nirmala et al.⁴⁰ and AbdelHamid et al.⁴⁴ have observed that the information related to thick blood vessels is more relevant in L3 detail subbands, whereas L1 has minimal information related to retinal structures.⁴¹ Consequently, for homogeneous retinal images, the information content is significantly larger for L3 than for L1 detail subbands. Nevertheless, abrupt illumination changes in nonhomogenous retinal images could result in increased information content in L1 detail subbands. Thus, homogenous retinal images tend to have higher WaveletEntropyL3/WaveletEntropyL1 ratios than nonhomogeneous images. In Fig. 7, examples of retinal images with varying homogeneities are given along with their horizontal WaveletEntropyL3/WaveletEntropyL1 ratio. The WaveletEntropyL3/WaveletEntropyL1 ratio calculated for the three detail subbands is also added to homogeneity feature set.

Fig. 5

Nonhomogeneous retinal (a) color image, (b) red channel, and (c) green channel from DR2.⁹

Fig. 6

Homogeneous (first row) and nonhomogenous (second row) retinal images from DR2.⁹ Then from left to right: color image, green and red channel wavelet entropies of the L3 horizontal, vertical, and diagonal subbands.

Fig. 7

Examples of (a) homogenous (from DR2)⁹ (b) nonhomogeneous sharp (from DR2)⁹ and (c) nonhomogenous blurred (from DRIMDB)³⁰ retinal images with WaveletEntropyL3/WaveletEntropyL1 ratios of (a) 37, (b) 11, and (c) 7.5.

The overall homogeneity feature set contains the $S_{\text{retina}}$ features (mean, variance, IQR, skewness, kurtosis, CV, $P_{\mathrm{mid}}$, energy, and entropy) along with the difference between red and green L3 wavelet entropies and the WaveletEntropyL3/WaveletEntropyL1 ratio.

4.4.

Field Definition

Sharp and well-illuminated retinal images can be regarded as medically unsuitable if they have missing or incomplete retinal structures. Inadequate field definition can be caused by patient movement, latent squint, or insufficient pupil dilation.²³ In this study, we are concerned with macula centered retinal images captured at 45 deg FOV commonly recommended in DR screening.^9,23,24 Macula centered retinal images should have the macula centered horizontally and vertically within the image with the OD to its left or right for left and right eyes, respectively.²³ Such an orientation assures the diagnostically important macular region is completely visible within the retinal images. Figure 8 shows examples of macula and nonmacula centered retinal images.

Fig. 8

Retinal images with different field definition (a) macula centered, (b)–(d) nonmacula centered from DR2.⁹

The proposed algorithm assesses an image’s field definition by checking whether or not the OD is located in its expected position. Initially, the middle region within the image is divided into three blocks: left, center, and right as shown in Fig. 9. A relatively large center region (50% of the image height) is considered to take into account cases in which the OD is slightly shifted upward or downward while adequate field definition is still maintained. The algorithm evaluates the image’s field definition by inspecting only these three blocks taking into account the following two observations:

Fig. 9

Regions of feature extraction in field definition evaluation algorithm (image from DR1⁹).

Based on these two observations, four measures are proposed for field definition validation

Macula centered images are expected to have the macula within their center block and the OD in one of the edge blocks (left or right). Thus, based on the two former observations, macula centered images will be characterized by ${\text{Sharp}}_{\max }$ and ${\mathrm{Illum}}_{\max }$ that are larger than ${\text{Sharp}}_{\text{center}}$ and ${\mathrm{Illum}}_{\text{center}}$, respectively. Furthermore, the difference features exploit the fact that the macula region, expected to be in the image’s center block, has lower vessel concentration and darker illumination relative to the block including the OD.

The field definition feature set includes the measures defined by Eqs. (6)–(9). The sharpness and illumination features were calculated using the L4 wavelet-based entropy and RGB features described in Secs. 4.1 and 4.2, respectively.

4.5.

Outliers

Outlier images can be captured during retinal image acquisition due to incorrect camera focus or false patient positioning.³⁰ They consist of nonretinal information making them irrelevant for medical diagnosis. If outlier images are processed by ARSS, they can be falsely classified as being medically suitable images leading to unreliable or incorrect diagnosis. Generally, retinal images are characterized by their unique coloration having a dominating reddish-orange background along with some dark red vessels and the yellowish OD. This characteristic dominating reddish-orange color of the retinal images can be explained by the RTM model detailed in Sec. 4.3.1. On the other hand, nonretinal images tend to have more generalized colorations as shown in Fig. 10.

Fig. 10

Outlier images from DRIMDB.³⁰

Color information from the RGB channels has been commonly used to discriminate retinal from nonretinal images.^19,30 Nevertheless, several color models exist that separate color and luminance information such as the CIELab and HSV models. In the CIELab model, images are represented by three color channels including image luminance, difference between red and green as well as difference between yellow and blue. In the HSV, image color, saturation, and value information are separated into three different channels.

In this work, color information is also used to differentiate between retinal and nonretinal images. Statistical features calculated from three different color models are compared for outlier image detection. The mean, variance, skewness, kurtosis, and energy calculated from the RGB channels, HSV hue and saturation channels, and the CIELab color channels will be compared in the next section to choose the color model more suitable for outlier features calculation.

5.1.

Sharpness Algorithm Classification Performance

In order to test the proposed sharpness algorithm, two datasets of different resolutions and degree of blur were used. The first dataset is a subset from DRIMDB and the second dataset is HRF. Each of the datasets consists of 18 good quality and 18 bad quality images. Unlike the severely blurred DRIMDB images, the unsharp images in the HRF dataset are only slightly blurred with the main vessels still visibly clear. Images of the DRIMDB and HRF datasets were resized to $448\times 512\text{pixels}$ and $960\times 1280\text{pixels}$, respectively. Image resizing was performed to make them suitable for five-level wavelet decomposition.

For the first dataset, classification using the complete sharpness feature set resulted in an area under the receiver operating characteristic curve (AUC) of 0.997. In further experiments, only the wavelet entropy, mean, IQR, or ${\text{WEntropy}}_{\mathrm{diff}}$ were used for sharpness classification. AUCs higher than 0.99 were achieved when any of the four sharpness measures derived from either the green, red, or both channels was separately used. Moreover, the elimination of L5 sharpness features did not degrade the results in any of the previous cases. The high classification performance can be attributed to the ability of the wavelet-based features to easily separate sharp images from severely blurred images.

The second dataset is considered more challenging since its poor quality images are only slightly blurred. An AUC of 0.951 was achieved using the entire sharpness feature set. Several observations can be made from further performed experiments whose results are summarized in Tables 2 and 3. First, all red channel sharpness features resulted in higher classification performance than green channel features. Nevertheless, the best results were achieved when both the red and green channels were used for sharpness feature computations. Second, Table 2 shows that a slight improvement in classification results (AUC of 0.954) is depicted upon the omission of L5 sharpness features when both red and green channel features are combined. Third, the comparison between absolute and ${\text{WEntropy}}_{\mathrm{diff}}$ classification results summarized in Table 3 shows an advantage to using ${\text{WEntropy}}_{\mathrm{diff}}$.

Table 2

AUC for the HRF dataset using wavelet-based sharpness features.

Table 3

Comparison between the AUCs using wavelet entropy versus WEntropydiff (L1 to L4) for the HRF dataset.

Based on the experiments performed on the two sharpness datasets, several modifications to the sharpness feature set were made. Previous research has shown that L5 scarcely had any sharpness-related information.^41,44 Furthermore, experiments have shown that the elimination of L5 sharpness features had minimal effect on classification performance. Consequently, only four-level wavelet decompositions will be employed in sharpness computations. Moreover, absolute wavelet entropy will be omitted from the sharpness vector as the ${\text{WEntropy}}_{\mathrm{diff}}$ performance was shown to be either equivalent or better than the absolute wavelet entropy. The final quality feature vector will thus include the wavelet-based mean, IQR, and ${\text{WEntropy}}_{\mathrm{diff}}$ sharpness features. All sharpness features are to be calculated from both the red and green channels of L1 to L4 detail subbands.

5.2.

Illumination Algorithm Classification Performance

A dataset consisting of 60 well-illuminated and 60 poorly illuminated images from DR1 was constructed to test the illumination algorithm. The poorly illuminated images had the overall image exposure issues of being either over or under illuminated. Images were resized to $432\times 496\text{pixels}$ to make them suitable for four-level wavelet decomposition. An AUC of 1 was achieved for the proposed illumination feature vector. Table 4 shows classification results when utilizing separate and combined color channels for feature computation. Higher results were obtained by the green channel features followed by the red then the blue. Superior performance was achieved when all color channel features were combined. The final quality feature vector will contain the complete illumination feature set.

Table 4

AUC for the illumination dataset using L4 wavelet illumination features.

5.3.

Homogeneity Algorithm Classification Performance

In order to test the proposed homogeneity algorithm, a dataset including 60 homogeneous and 60 nonhomogeneous retinal images from DR1 was created. Homogeneity classification results using the different proposed features are summarized in Table 5. Classification with $S_{\text{retina}}$ features, wavelet entropy R–G, and WaveletEntropyL3/WaveletEntropyL1 resulted in AUCs of 0.981, 0.899, and 0.982, respectively. The complete homogeneity feature set resulted in AUC of 0.999. All proposed homogeneity features will be added to the final quality vector.

Table 5

AUC for the homogeneity dataset using Sretina and wavelet features.

5.4.

Field Definition Algorithm Classification Performance

A dataset consisting of 60 macula centered and 60 nonmacula centered retinal images was constructed from DR2 for assessment of the proposed field definition features. All images were resized to $432\times 496\text{pixels}$ to make them suitable for four-level wavelet decomposition. Results using the proposed field definition features resulted in a perfect classification with an AUC of 1.

5.5.

Outlier Algorithm Classification Performance

In order to test the outlier detection algorithm, the DRIMDB dataset was used, which includes 194 retinal images and 22 nonretinal images. The RGB channels, HSV model’s hue and saturation channels, and CIELab model’s a (red–green) and b (blue–yellow) channels were compared for feature computation. As illustrated in Table 6, the lowest classification results were attained by the RGB features giving an AUC of 0.968. The HSV and CIELab features resulted in slightly higher AUCs of 0.993 and 0.991, respectively. Based on classification results, only the outlier features derived from the HSV color space will be included in the final quality vector.

Table 6

AUC for DRIMDB outlier classification using RGB, HSV, and CIELab features.

5.6.

Overall Quality Classification Results

In Sec. 5.1–5.5, each of the five different quality feature sets for sharpness, illumination, homogeneity, field definition, and outlier assessment were separately tested and evaluated. Based on performed analyses, the overall quality feature vector is composed from features described in Table 7. In this section, the DR1 dataset is used for the evaluation of the overall quality feature vector. It is important to note that images from the DR1 dataset were not utilized in any analysis that could affect or bias the final classification results. The good quality images within the DR1 demonstrate varying degrees of sharpness and illumination. Moreover, the DR1 bad quality images suffer from a wide range of quality issues including slight/severe blur, over/under illumination, nonhomogeneity, nonmacula centering, and nonretinal images.

Table 7

Overall quality feature vector description.

Several preprocessing steps were performed on the DR1 dataset prior to feature extraction. Initially, a $5\times 5$ median filter was applied to all images for noise reduction. Next, unsharp masking and contrast adjustment were utilized to enhance the overall sharpness and contrast of the images, respectively. Generally, retinal image enhancement is especially useful for retinal images of borderline quality. Image enhancement improves the quality of these images further differentiating them from bad quality images. Furthermore, all images were resized to $432\times 496\text{pixels}$ to make them suitable for four-level wavelet decomposition.

Table 8 compares classification results of the proposed RIQA algorithm to other existing techniques. Two generic RIQA algorithms were reimplemented by the authors for the sake of this comparison. The first algorithm was introduced by Fasih et al.¹⁷ who combined the cumulative probability of blur detection (CPBD) sharpness feature⁶⁰ with run length matrix (RLM) texture features⁶¹ for quality assessment. Fasih et al. calculated their features only from the OD and macular regions of the retinal image. The second algorithm is the work of Davis et al.¹⁵ who computed spatial frequency along with CIELab statistical and Haralick textural features⁶² from seven local regions covering the entire retinal image. Moreover, the results of the hybrid approach by Pires et al.⁹ are also included in the comparison. Pires et al. used the area occupied by retinal blood vessels, visual word features along with quality assessment measures estimated from comparing the test image to both blurred and sharpened versions of itself.

Table 8

AUC results comparing proposed clarity and content RIQA algorithm with other algorithms for DR1.

An SVM classifier with an radial basis function (RBF) kernel was used for the classification of the proposed and reimplemented algorithms. Generally, the performance of the SVM classifier is dependent on its cost and gamma parameters. A grid search strategy was performed to find the optimal classifier parameters in each case. Among the previous works, best classification results were obtained by Pires et al.’s hybrid algorithm achieving an AUC of 0.908. Nevertheless, the results of the proposed RIQA algorithm exceed all the methods in comparison with an AUC of 0.927.

6.1.

Sharpness Algorithm Analysis

Wavelet decomposition has the advantage of separating image details of different frequencies in subsequent wavelet levels. Generally, wavelet-based sharpness features were shown to have the advantages of being simple, computationally inexpensive, and considering structural information while giving reliable results for datasets of varying resolutions and degree of blur.

Several wavelet-based features were computed from the red and green image channels for sharpness assessment. The green channel is often used solely for RIQA due to its high blood vessel contrast. However, the visibility of the retinal structures within the red channel can also be affected by the image’s overall sharpness. Furthermore, the OD is generally more prominent in the red channel than in the green channel of retinal images.⁶³ Hence, combining the red and green channels for sharpness assessment allows for better consideration of the different retinal structures. In this work, it was shown that both green and red channels were relevant to retinal image sharpness assessment and that their combination can improve the classification performance. The usefulness of the retinal images’ red channel has been previously depicted in segmentation and enhancement algorithms. In OD segmentation, the red channel has been used either separately⁶³ or alongside the green channel⁶⁴ since the OD boundary is more apparent in the red channel. In retinal image enhancement, information from the red channel was used along with the green channel to improve results.⁶⁵

Generic RIQA algorithms commonly use a combination of sharpness, textural, and statistical features. Spatial frequency¹⁵ and CPBD¹⁷ are among the sharpness features used in generic methods. Classification of the sharpness dataset drawn from DRIMDB using spatial frequency and CPBD resulted in AUCs of 0.948 and 0.81, respectively. Wavelet-based sharpness features were shown in Sec. 5.1 to give an AUC higher than 0.99 for the same dataset showing the superiority of the wavelet-based sharpness measures.

Earlier research has shown that there is a relation between the different wavelet levels and the various retinal structures.⁴¹ In order to study the wavelet level significance for the different wavelet-based sharpness features, the proposed features were computed from individual levels and results are summarized in Table 9. Classifications were performed by the kNN classifier using features calculated from both the red and green channels of the HRF dataset resized to $960\times 1280\text{pixels}$. In the given case, results show that L3 features produced the highest results for wavelet entropy, whereas L4 features gave the highest results for mean and IQR. Consequently, the wavelet level significance is found to be feature dependent. The approach adopted in this work combines sharpness features from different wavelet levels instead of finding the most relevant levels. For each of the three sharpness features, best results were achieved when L1 to L4 were combined. Hence, combining sharpness features from several wavelet levels was found to give more consistent classification performance than relying on a single-wavelet level for the feature calculation.

Table 9

AUC for sharpness classification using different individual and combined wavelet levels on the HRF dataset.

6.2.

Illumination and Homogeneity Algorithms Analysis

Modern theories on color vision explain that the human eye views color in a two-stage process described by the trichromatic and opponent theories.⁶⁶ The trichromatic theory states that the retina has three types of cones sensitive to red, green, and blue. The opponent theory indicates that somewhere between the optic nerve and the brain, this color information is translated into distinctions between light and dark, red and green, as well as blue and yellow. RGB and CIELab color models are the closest to trichromatic and opponent theories, respectively. Nevertheless, other models such as HSV and HSI also separate the image color and illumination information.

In this work, RGB illumination features were computed from the approximation wavelet subbands since the image’s illumination component resides within its wavelet approximation subband.⁶⁷ The utilization of the RGB model for retinal illumination feature calculation is compared to using the luminance information of the CIELab, HSV, and HSI models. For the previously created illumination dataset, the mean and variance were computed from the four color models. Classification results using kNN are summarized in Table 10. Initially, only the illumination channels were used for classification resulting in AUCs of 0.953, 0.932, and 0.944 for the CIELab, HSV, and HSI color models, respectively. Next, the color and saturation channel features were added resulting in an increase in AUCs by approximately 4% to 6% for the different color models to become 0.992, 0.987, and 0.99 for the CIELab, HSV, and HSI color models, respectively. Hence, it is demonstrated that utilizing all color channels improves the illumination classifier performance for all the color models. The proposed wavelet-based illumination features achieve an AUC of 1 hence demonstrating competitive performance as compared to other color models.

Table 10

AUC for the illumination dataset using the RGB, CIELab, HSV, and HSI color models.

Retinal image homogeneity was assessed using a combination of measures that were especially customized for retinal images. The proposed $S_{\text{retina}}$ channel was inspired by the saturation channel of the HSV model ($S_{\mathrm{HSV}}$) along with the RTM. The previously created homogeneity dataset was used to compare the homogeneity features computed from $S_{\text{retina}}$, $S_{\mathrm{HSV}}$, and $S_{\mathrm{HSI}}$ channels. Table 11 summarizes the results performed by the kNN classifier in which an AUC of 0.981 was achieved by the proposed $S_{\text{retina}}$ features, which is 5% and 9% higher than the AUCs from $S_{\mathrm{HSV}}$ and $S_{\mathrm{HSI}}$, respectively. These results indicate the efficiency of the proposed $S_{\text{retina}}$ channel for retinal image homogeneity assessment as compared to saturation channels from different color models.

Table 11

AUC for the homogeneity dataset using different saturation channels.

The most previous work in RIQA did not discriminate between overall image illumination and homogeneity. Instead, various statistical and textural features were combined to consider different retinal image luminance aspects. For example, Yu et al.⁷ used seven RGB histogram features (mean, variance, kurtosis, skewness, and quartiles of cumulative distribution function) to measure the retinal image brightness, contrast, and homogeneity. They also added entropy, spatial frequency, and five Haralick features to measure image complexity and texture. Davis et al.¹⁵ used the kurtosis, skewness, and mean of the three CIELab channels for complete characterization of retinal image luminance. In order to compare retinal image luminance assessment algorithms, the illumination and homogeneity datasets were combined. Table 12 compares the kNN classifier results of the proposed illumination and homogeneity features to the measures suggested by Yu et al.⁷ and Davis et al.¹⁵. AUCs of 0.985, 0.933, and 0.943 were achieved for the proposed features, the RGB features,⁷ and the CIELab features,¹⁵ respectively. The AUC of the algorithm by Yu et al.⁷ was increased by $\sim 4\%$ to become 0.975 when texture and complexity features were added. Nevertheless, the introduced features achieved the highest classification results with an AUC of 0.985. Overall, analyses summarized in Tables 10–12 show that the newly proposed illumination and homogeneity features are of high relevance to retinal image luminance assessment.

Table 12

Comparison between luminance features based on AUC for combined illumination and homogeneity datasets.

Studies have shown that retinal structures captured within color images as well as the phenotype and prevalence of some ocular diseases varied with ethnicity.^68,69 For example, pigmentation variations among ethnic groups can affect the observed ocular image.⁷⁰ Giancardo et al.⁷¹ have noticed that color retinal images of Caucasians have a strong red component whereas those of African Americans had a much stronger blue component. This could have an effect on the proposed retinal saturation channel when dealing with different ethnic groups. A possible solution to this issue could be the introduction of a parameterized saturation channel that could be adjusted according to the ethnic group under consideration. In conclusion, the study of the effect of ethnicity on retinal image processing techniques is essential to ensure reliability of ARSS for heterogeneous population screening.

6.3.

Field Definition and Outliers Algorithms Analysis

RIQA algorithms usually classify retinal images into good or bad quality images relying solely on image clarity features assuming processed images to be retinal and having adequate field definition. In real systems, such an assumption cannot be made in which images with inadequate content can be captured. Fleming et al.²³ have observed that $\sim 50\%$ of medically unsuitable retinal images fail due to inadequate field definition. Further processing of images with inadequate content by automatic RIQA systems could result in misdiagnosis. Thus, validating retinal image content along with its clarity is an indispensable step in RIQA specifically in automated systems.

Macula-centered retinal images are commonly considered in field definition assessment algorithms targeting DR screening systems.^9,23,24 The macula is the part of the retina responsible for detailed central vision. DR signs thus become more serious if located near the center of the macula.⁷² Hence, the macula and surrounding regions are specifically important in the diagnosis of DR. In this study, we were concerned with macula centered retinal images for DR screening. Nevertheless, RIQA is an application specific task.²² Different considerations in the RIQA algorithm might be necessary if different FOV or retinal diseases were of interest.

The introduced field definition assessment algorithm operates by checking that the OD is in its expected position. It is based on the observations that the OD has a density of thick vessels as well as being overly illuminated. Therefore, the introduced algorithm exploits the proposed wavelet-based sharpness and illumination features. Comparing L3 and L4 for field definition feature calculation resulted in AUCs of 0.991 and 1, respectively. Thus both levels were found closely significant for field definition assessment. Finally, image color information was employed to differentiate between retinal and nonretinal image. Comparison between features calculated from the RGB, CIELab, and HSV has shown that generally color information is adequate for outlier detection.

6.4.

Overall RIQA Algorithm Analysis

In this work, RIQA was performed based on the combination of five quality feature sets used to evaluate image sharpness, illumination, homogeneity, field definition, and outliers. The proposed quality algorithm relies on the computation of several transform and statistical-based features. Generally, transform-based approaches have the advantage of being consistent with the human visual system⁷³ stating that the eye processes high- and low-frequencies through different optical paths.⁷⁴ Furthermore, both transform and statistical-based features have the advantage of being computationally inexpensive while giving reliable results.

Initially each of the five feature sets was separately evaluated using custom datasets. Classification results showed the efficiency of the presented features in which all the feature sets achieved AUCs greater than 0.99 (with the only exception of the sharpness features applied to the slightly blurred HRF dataset giving a maximum AUC of 0.954). Finally, all proposed clarity and content RIQA features were combined for the classification of the DR1 dataset achieving an AUC of 0.927 which is between $\sim 2\%$ to 4.5% higher than other RIQA algorithms from the literature.

The overall RIQA algorithm required $\sim 2.5\mathrm{s}$ for preprocessing and overall feature calculation of a single $432\times 496\text{pixels}$ image using MATLAB. Several methods can be adopted to further reduce the overall processing time of the algorithm. The implementation of the algorithm using more efficient programming languages can help enhance the run time. Parallel processing of the feature computations as well as feature selection algorithms can also reduce the execution time. Furthermore, the DecWT can be used for image decomposition instead of UWT.

DecWT is characterized by its subsampling nature resulting in halving of image size with each level of decomposition. As a result, DecWT has the advantages of being fast and storage efficient. However, DecWT is shift variant such that any modification of its coefficients results in the appearance of a large number of artifacts in the reconstructed image.⁷⁵ UWT was designed to overcome the DecWT limitation by avoiding image shrinking with multilevel decomposition leading to the preservation of more relevant image information.⁴⁸ Furthermore, UWT is shift invariant, making it more suitable for wavelet-based image enhancement and denoising than DecWT.³⁹

The advantages of UWT over DecWT come with the trade-off of increased processing time. Comparison between UWT and DecWT timing performance for the proposed algorithm is summarized in Table 13. DecWT is shown to be more than five times faster than UWT for wavelet decomposition and fifteen times faster in sharpness wavelet entropy calculation. Overall, the preprocessing and final quality feature vector computation was found to be approximately four times faster when DecWT was used instead of UWT for image decomposition. Furthermore, classification performance was similar whether DecWT or UWT was used for image decomposition (AUC of 0.927 in both cases for RBF kernel in SVM). Thus, the proposed wavelet-based features are shown to be equally efficient when calculated from either the decimated or undecimated wavelet decompositions. In conclusion, for the proposed RIQA algorithm, DecWT is more suitable for real-time systems as it is more computationally efficient.

Table 13

Timing comparisona in seconds between UWT and DecWT in RIQA.