1.

Introduction

Optical coherence tomography (OCT)^1,2 is a relatively new imaging modality, which was introduced to overcome the limitation of previous imaging techniques as it provides high resolution cross-sectional images of coronary arteries. In contrast to intravascular ultrasound, OCT is based on the analysis of reflected light, instead of acoustic waves, allowing visualization of plaque features with a higher analysis (axial resolution: 12 to 18 μm, lateral resolution: 20 to 90 depending on the depth of focus).³ The unique imaging resolution of OCT has drawn the attention of the scientific community from its early stages. However, its clinical applications were initially limited due to the fact that imaging required blood removal. This drawback was addressed with a proximal balloon occlusion⁴ which, however, caused ischemia and possibly vessel wall trauma. A later approach used saline injection for blood removal but the slow image acquisition speed restricted OCT imaging in short segments. The recent introduction of the second-generation OCT systems, the so-called frequency domain OCT (FD-OCT),¹ permits fast image acquisition (up to $160\mathrm{frames}/\mathrm{s}$) and fast pull-back speeds allowing thus visualization of long segments within few seconds.⁵ These systems increased considerably the applicability of OCT and today this modality constitutes an indispensable tool in the clinical and research arena.

OCT allows evaluation of the luminal dimensions, assessment of vessel wall morphology and identification of features associated with increased vulnerability, such as the thickness of fibrous cap, microcalcifications, neo-vessels, and the presence of macrophages, which cannot be evaluated by other intravascular imaging modalities.^6,7 OCT has limited penetration,⁶ and quite often does not allow imaging of the entire vessel wall, especially in segments with large lumen and increased plaque burden. However, its high resolution permits detailed evaluation of the superficial plaque and nowadays this modality is considered as the gold standard for the in vivo characterization of plaque. According to the recently published expert consensus,^6–8 four different tissue types can be detected in OCT: calcium (CA), that appears as a signal poor region with sharply delineated borders, fibrous tissue (FT), that is portrayed as a high back-scattering homogeneous area, lipid tissue (LT), that is a shown as a signal poor region with diffuse-delineated borders with a fast drop-off and little or no back-scattering signal, and mixed tissue (MT), that features characteristics from multiple tissue types.

Traditionally, OCT lumen detection and plaque characterization^9,10 was performed manually. However, this process is time consuming and can be unreliable in the hands of nonexperienced operators. To overcome these drawbacks and enhance the clinical and research applications of OCT, several methodologies have been developed for the automated processing of OCT images.^11,12 The most recent ones were proposed by Ughi et al.¹³ and Tsantis et al.¹⁴ who presented an automated methodology that permitted detection of the lumen borders and the stent struts, allowing evaluation of their apposition and coverage. Xu et al.¹⁵ were the first who attempted to develop an automated plaque characterization methodology. They used histological images as gold standard and studied the backscattering and attenuation coefficient in three tissue types (CA, LT, and FT), aiming to determine a cutoff value for each tissue. Similarly, the methodology proposed by van Soest et al.¹⁶ examined the attenuation coefficients of healthy segments, intimal thickening plaques, LT and fibrous caps infiltrated by macrophages. However, both approaches failed to define fixed cutoff values of the backscattering and attenuation coefficients that would allow reliable plaque characterization. In addition, both approaches analyzed the backscattering and attenuation coefficients across the entire A-lines without being able to identify the plaque region. Wang et al.¹⁷ proposed a semiautomated methodology for detecting CA. They segmented the lumen, applied edge detection to detect roughly the CA borders, and then, by applying active contours, they detected the CA borders. However, the methodology had one major limitation: the occurrence of many false-positive CA which have to be removed by the user manually to apply clinical measures. Although the authors have proven that image processing techniques can be applied to detect CA they did not present a plaque characterization methodology.

In this study, for the first time, we introduce an automated methodology for the segmentation of the lumen border and the characterization of the composition of the superficial plaque in OCT images. The methodology proposed for the identification of the different tissue types relies on the combination of medical knowledge, image processing, and machine learning techniques and is able to process high-quality OCT images in a fully automated manner without user interaction. The innovative aspects of the proposed methodology are:

The proposed approach was validated against the estimations of expert observers in real patient data. Our results indicate that our methodology can accurately detect luminal borders and characterize the composition of superficial plaque.

2.

Dataset

Twenty-seven OCT pullbacks from 22 patients, who underwent OCT investigation for clinical purposes after giving consent, were used to train and validate the proposed methodology. The images were acquired by a C7XR Fourier Domain system (Lightlab Imaging, Westford, Massachusetts) with the use of an automated pull-back device under continuous injection of contrast medium.

3.

Materials and Methods

Four steps are followed in order to characterize the composition of the plaque: image acquisition, detection of the TA, TA segmentation, and plaque characterization. Each one of these stages is described in detail in the following sections and is shown schematically in Fig. 1.

Fig. 1

Schematical representation of the proposed plaque characterization methodology.

3.2.

Detection of the TA

A region in each OCT frame is determined for the plaque characterization methodology to be applied. This region is called TA and is located between the lumen border and the expanded (up to 1.5 mm) lumen border (outer border of the TA). The TA detection stage includes two steps: the detection of the lumen border and the extraction of the outer border of the TA.

3.2.1.

Lumen detection

Lumen border ($L_{mn}$) detection is performed in the polar OCT images [$I(r,\theta )$] [Fig. 2(a)–2(c)] and includes the following steps:

• 1. Apply a Gaussian filter on the initial image [$I(r,\theta )$].

• 2. Perform Otsu’s automatic thresholding method¹⁹ in order to reveal binary objects and remove the catheter pixels.

• 3. Remove the 8-connected neighborhood nonzero objects (artifacts).

• 4. Remove the object close to the catheter with area less than $1/5$ of the image area and mean intensity [of the objects’ pixels that correspond to $I(r,\theta )$ image] higher than 200. This area belongs to the catheter artifact area $A_{\mathrm{cath}}$.

• 5. Scan each column of the image from top to bottom, save the first nonzero pixel of the column, and connect the nonzero pixels in order to find the lumen contour.

• 6. Transform the image from polar to Cartesian coordinates.

Fig. 2

Detection of the lumen border and extraction of the tissue area (TA): (a) Initial image in polar coordinates, (b) catheter and catheter artifact removal, (c) the lumen border in polar coordinates, and (d) lumen border marked with blue on the reconstructed OCT image. To extract the outer border of the TA, radii are drawn toward the lumen border and extended by 1.5 mm, the curve defined by the distal end of the radius corresponds to the outer border of the TA (yellow curve).

3.2.2.

Detection of the Outer Border of the TA

The gravitational center of the lumen border is determined in each OCT frame and then 360 radii (with 1-deg interval) are drawn toward the lumen border and expanded 1.5 mm distally to the detected border. Our decision to focus on the superficial plaque was based on the small tissue penetration of the intravascular OCT systems (1 to 2 mm) and on the fact that there is a decreased signal-to-noise ratio in the pixels, which depict tissues that are deeply embedded in the plaque.⁶ The curve that connects the distal edge of the drawn radius defines the outer border of the TA ($L_{mn}^{\mathrm{expan}}$) [Fig. 2(d)].

3.3.

TA Segmentation

TA pixels belong either to plaque or to extravascular tissue. To discriminate the pixels that belong to the plaque from those that image extravascular tissue, a $K$-means algorithm is implemented. This approach appeared to outperform other methods that were tested (automatic thresholding, border detection) in detecting the plaque area and was capable to identify the CA TA. A $K$-means algorithm partitions a given set of $N$ observations $\{x_1,\dots ,x_N\}$, into a fixed number of $K$ clusters $S=(S_1,S_{2,}\dots ,S_K)$ in order to minimize the within cluster sum of squares:²⁰

Eq. (1)

$$\underset{S}{\arg \min }\sum _{l=1}^K\sum _{x_l\in S_{\iota }}{\Vert x_l-{\mu }_{\iota }\Vert }^2,$$

where ${\mu }_{\iota }$ is the mean of points in $S_{\iota }$.

The pixels corresponding to TA are classified to $K=3$ clusters [Fig. 3(b)] ($K=2$ and 4 clusters were also tested) resulting to a segmented image $I_{\text{segm}}(i,j)$ (Appendix A). The pixels that were classified in the first two clusters ($K_1$ and $K_2$) have an intermediate signal intensity and are likely to portray either the superficial LT, FT, and MT while the pixels classified in the third cluster $(K_3)$ have a low signal intensity and are likely to portray either the area behind the catheter artifact ($A_{\mathrm{B}\mathrm{cath}}$), or CA, or a signal poor region corresponding to deeply located LT or extravascular tissue.

Fig. 3

(a) Initial image, (b) segmented image using the $K$-means clustering method, blue and yellow pixels belong to lipid tissue (LT), fibrous tissue (FT), and mixed tissue (MT). The light blue pixels belong to: the area behind the catheter artifact, CA, extravascular tissue, or the deeper layers of LT, and (c) the calcium detected in the initial image is marked with white color.

3.4.

Plaque Characterization

Once the TA is segmented, the different plaque types can be detected.

3.4.1.

Calcium detection

CA appears as a signal-poor region with sharply delineated borders. After applying the $K$-means segmentation algorithm, the sharply delineated borders belong either to $K_1$ or $K_2$ cluster while the signal poor region located between these clusters to $K_3$ cluster [Figs. 3(a) and 3(b)]. Therefore to find the CA, each radius of the $I_{\mathrm{segm}}(i,j)$ [Fig. 3(b)] is scanned from the lumen border to the outer border of the TA. The radial segment which is clustered to $K_3$ and is enclosed by pixels that belong to the cluster $K_1$ or $K_2$ is labeled as calcium [Fig. 3(c)]. The above procedure is shown schematically in Fig. 3.

3.4.2.

Feature extraction

For each pixel of the area that corresponds to clusters $K_1$ and $K_2$ [Fig. 3(b)], a set of texture and intensity-based features is calculated in a $n\times n\text{pixel}$ neighborhood (Appendix B). Local binary patterns (LBPs) and gray level co-occurrence matrices (GLCMs) are the texture features which are employed.

The GLCM²¹ is a matrix with dimensions $L\times L$, defined over an image $I(i,j)$ that calculates how often a pixel with gray-level value $i$ (grayscale intensity) occurs spatially adjacent to a pixel with value $j$. The spatial relationship of the pixels is defined by an offset $(D_x,D_y)$ as

Eq. (2)

$$D_x=D·\cos (\theta ),D_y=D·\sin (\theta ),$$

where $\theta$ is the spatial direction and $D$ is the Euclidean distance between the pixels. For the proposed methodology, we compute the co-occurrence matrices $C_{(D_x,D_Y)}(i,j)$ using all the possible values of $\theta$ $[\theta =(\mathrm{0,45,90,135,180,225,270,315}\mathrm{deg})]$ and $D=2$ ($D=1$, 3 were also tested). Thus, eight $C_{(D_x,D_Y)}(i,j)$ are computed for each pixel in a $n\times n$ neighborhood. For each co-occurrence matrix $C_{(D_x,D_Y)}(i,j)$, the following measures are calculated:

Contrast:

Eq. (3)

$$f_{\text{contrast}}^{\theta }=\sum _{i=1}^L\sum _{j=1}^L(i-j)C_{(D_x,D_y)}(i,j).$$

Correlation:

Eq. (4)

$$f_{\text{correlation}}^{\theta }=\frac{\sum _{i=1}^L\sum _{j=1}^L[(i·j)C_{(D_x,D_y)}]-{\mu }_i·{\mu }_j}{{\sigma }_i·{\sigma }_j},$$

where ${\mu }_i=\sum _{i=1}^{L}i{C}_{(D_x,D_y)}(i,j)$ and ${\sigma }_{\iota }^2=\sum _{i=1}^L{i}^2{C}_{(D_x,D_y)}(i,j)-{\mu }_{\iota }^2$.

Energy:

Eq. (5)

$$f_{\text{energy}}^{\theta }=\sum _{i=1}^L\sum _{j=1}^L{[C_{(D_x,D_y)}(i,j)]}^2.$$

Homogeneity:

Eq. (6)

$$f_{\text{homogeneity}}^{\theta }=\sum _{i=1}^L\sum _{j=1}^L\frac{C_{(D_x,D_y)}(i,j)}{1+|i-j|}.$$

Therefore, 32 features are computed ($f_{\mathrm{glcm}}^1,f_{\mathrm{glcm}}^2,\dots f_{\mathrm{glcm}}^{32}$), four features for each angle $\theta =(0,45,90,135,180,225,270,315\mathrm{deg})$.

LBPs²² determine uniform texture patterns of $P$ members into circular neighborhoods with radius $R$. Among the various LBP tested, we used the rotation invariant ${\mathrm{LBP}}_{P,R}^{\mathrm{riu}2}$:²³

Eq. (7)

$${\mathrm{LBP}}_{P.R}^{\mathrm{riu}2}=\{\begin{array}{ll}\sum _{p=0}^{P-1}s(g_p-g_c),& \text{if}U({\mathrm{LBP}}_{P,R})\le 2\\ P+1,\hfill & \text{otherwise}\hfill \end{array},$$

where $g_c$ is the intensity of the center pixel and $g_p$ are the intensities of the circularly symmetric neighborhood, superscript ri refers to “rotation invariant” while $u2$ means that the pattern has $U$ values ($U$ is the number of spatial transitions in the pattern) of at most 2, $U({\mathrm{LBP}}_{P,R})=|s(g_{P-1}-g_c)-s(g_0-g_c)|+\sum _{p=1}^{P-1}|s(g_p-g_c)-s(g_{p-1}-g_c)|$ whereas the function $s(x)$, is defined as

$$s(u)=\{\begin{array}{cc}1,& u\ge 0\\ 0,& u<0\end{array}.$$

${\mathrm{LBP}}_{P,R}^{\mathrm{riu}2}$ output is a fixed set of discrete values: $0,1,2,\dots ,P+1$. $P=8$ and $R=1$ are used in the proposed methodology ($P=4$, $R=1$, $P=12$ $R=1.5$, and $P=16$, $R=2$ were also tested but the results were suboptimal) hence 10 LBP features $f_{\mathrm{lbp}}^0$, $f_{\mathrm{lbp}}^1,\dots ,f_{\mathrm{lbp}}^9$ are computed.

The above 42 features (32 GLCMs and 10 LBPs) along with the entropy ($f_{\mathrm{en}}$) and mean value ($f_{\mathrm{mv}}$)²⁴ of the $n\times n$ neighborhood result to a set of 44 features for each pixel belonging into the two clusters: $K_1$ and $K_2$.

3.4.3.

Feature selection

Many factors affect the accuracy and success of classification algorithms on a given task. One of these factors is the redundant information given in the classifier. Therefore, feature selection is crucial for improving the accuracy of the proposed plaque characterization methodology. We have employed the wrapper feature selection (WRP)²⁵ to find the best features for the proposed methodology. The WRP method produces feature subsets from all possible feature combinations. Afterward, all the possible feature combinations are used as input to a target classification algorithm, and the overall accuracy of each combination is estimated. The feature subset that achieves the highest classification accuracy is chosen as the best feature combination for the specific classification algorithm (Appendix C).

3.4.4.

Classification

Once the features are extracted, the pixels belonging to the clusters $K_1$, $K_2$ are classified to one of the remaining three plaque types: LT, FT, and MT. For this purpose, different classification algorithms were tested [support vector machines, neural networks, and random forests (RF)]. The RF provided the best classification results and thus this classification algorithm was used by our methodology. The RF²⁶ is an ensemble classifier that consists of many decision trees; RF selects randomly the number of features in order to construct a collection of decision trees with controlled variation. Several numbers of features and trees were tested to find the best combination for the RF classifier (Appendix C).

3.4.5.

Postprocessing

In the available IV-OCT imaging configurations, the optical properties of LT limit the ability of OCT to image the full extent of lipid-rich plaques. Since the $K$-means algorithm segments only the pixels portraying FT, MT, and superficial LT, the pixels that portray deeper layers of LT cannot be classified (Fig. 4). When experts manually annotated LT tissue, they approximate the outer border of LT. In brief, they identify the external elastic lamina and draw an arc that connects its lateral extremities [Fig. 4(b)]. Following the same rationale with experts, we automatically estimate the outer border of LT implementing the following approach:

• 1. Transform the segmented image [$I_{\text{segm}}(i,j)$] into polar coordinates [$I_{\text{Psegm}}(i_p,j_p)$].

• 2. Scan each column of image [$I_{\text{Psegm}}(i_p,j_p)$] from the $L_{mn}^{\mathrm{expan}}$ to $L_{mn}^{\mathrm{expan}}-{\mathrm{Th}}_{\text{post}}$ and save the first zero pixel of the $I_{\text{segm}}(i,j)$.

• 3. The area between the connected zero pixels and the segmented region is considered as LT plaque.

• 4. Transform the image from polar to Cartesian coordinates.

Fig. 4

The postprocessing step of the proposed methodology: (a) initial image, (b) expert limitation on detecting the outer border of the LT plaque, (c) the initial image with the segmented region marked with white color, (d) the expanded lumen border transformed to polar coordinates, the distal border of the TA (yellow line) is reduced to 0.75 mm and a new line is drawn (yellow dashed line) in order to detect the outer border of the LT, (e) the image characterized before applying the postprocessing step, and (f) the image characterized after applying the postprocessing step, (CA: white, LT: red, FT: dark green, and MT: light green).

The above described approach is shown in Fig. 4, while the sensitivity analysis for applying the postprocessing threshold (${\mathrm{Th}}_{\mathrm{post}}=0.75\mathrm{mm}$) is described in Appendix D.

4.

Validation Methodology

From our data set (27 OCT pullbacks), 556 OCT images were randomly selected and one expert annotated the lumen borders. Images portraying stented segments as well as frames with a minimal thrombus burden or a small amount of residual blood that did not affect image quality were included in the validation set. On the other hand, frames portraying malapposed struts or having an increased thrombus burden or residual blood in the lumen were excluded from the analysis. We also excluded frames having motion artifacts.⁷ To validate the performance of the proposed segmentation methodology, we compared the expert’s estimations with those derived by our approach.

Furthermore, two experts selected 200 images with no artifacts that portrayed diseased coronary segments, identified the presence of tissues with a typical morphology, and detected the CA, LT, FT, and MT.^6–8 In this analysis, MT was defined as the tissue that has features noted in CA and LT. Any disagreements in experts’ annotations were resolved by consensus. Sixty images were randomly selected and used to train the classification algorithm and the rest to validate the performance of the proposed methodology. From the training set of 60 images, a fully balanced dataset of 12,000 pixels was created by randomly selecting 4000 pixels from each one of the annotated plaques LT, FT, and MT. In the 140 images that were used for testing the methodology, 150 tissue regions with a typical morphology were annotated (34 CA, 36 LT, 47 FT, and 33 MT) by the two experts, and their mean estimation was used as gold standard to validate the proposed methodology.

5.

Results

An excellent agreement was noted for the lumen, CA, FT, LT, and MT areas estimated by our approach and the experts (Table 1). The correlation coefficient analysis and the Bland-Altman plots for the lumen areas are shown in Fig. 5. Figure 6 shows the correlation plots between the areas estimated by our approach and the areas estimated by the two experts for the four tissue types (CA, LT, FT, and MT), while Fig. 7 shows the results of the Bland-Altman analysis.

Table 1

Results of the proposed methodology using different validation metrics.

Fig. 5

(a) Correlation plot and (b) Bland-Altman plots for the lumen areas estimated by our methodology and the expert.

Fig. 6

Correlation plots for the (a) CA, (b) LT, (c) FT, and (d) MT areas estimated by our methodology and the two experts.

Fig. 7

Bland–Altman plots for the (a) CA, (b) LT, (c) FT, and (d) MT areas estimated by our methodology and the two experts.

The qualitative validation metrics showed an excellent agreement between our methodology and the estimations of the expert for the lumen areas (overlapping area ratio: 0.99, nonoverlapping area ratio: 0.02). The overlapping area analysis for the tissue types demonstrated that our approach classifies correctly most of the pixels detected by the two experts that portray tissues with a typical morphology (overpaying area ratio: 0.81 for the CA, 0.71 for the LT, 0.87 for the FT, and 0.81 for the MT). The disagreement between our approach and the mean estimations of the experts was low for the CA and FT tissue (nonoverlapping area metric: 0.35 and 0.37, respectively). On the other hand, for the LT and the MT the nonoverlapping area ratio was 0.51 and 0.47, respectively, indicating that the position of the detected tissue within the vessel wall is not exactly the same. These discrepancies noted in the LT and MT are due to the limited ability of our methodology to accurately detect their lateral extremities and to fact that the distal border of both tissues is not visible in OCT. Examples of the proposed methodology in detecting the lumen borders are shown in Fig. 8, while Fig. 9 presents application examples of the proposed methodology in detecting the four tissue types.

Fig. 8

Results of the proposed lumen detection step in (a) a native segment, (b) in a stented segment, (c) in a segment having thrombus, and (d) in a segment having residual blood. The lumen border is marked with blue.

Fig. 9

Application examples of the proposed methodology: (a) Initial images, (b) annotations over the initial images (white-CA, red-LT, green-FT, and light green MT) over the initial image, (c) color-coded images produced by our methodology using the initial images (white-CA, red-LT, green-FT, and light green MT).

6.

Discussion

In this work, a fully automated luminal border detection and plaque characterization methodology is presented that allows expedited analysis of the FD-OCT data. In the proposed methodology, the luminal borders are extracted and used to define the TA. On the extracted TA, segmentation techniques are applied, initially differentiating the extravascular tissue from the plaque, and then four tissues types are detected, namely: CA, LT, FT, and MT.

The two methodologies that have been presented in the past for characterizing the type of the plaque in FD-OCT data^15,16 relied on the analysis of the reflected light and attempted to correlate the attenuation coefficients with various plaque formations. They both focused on the separate processing each A-line and used the attenuation coefficient of the light to correlate each individual line with a specific plaque type. However, these techniques do not combine information provided by adjacent A-lines to classify more accurately plaque components and failed to define a cutoff value for each plaque type so as to develop a precise and complete plaque classification methodology.

On the contrary, the methodology presented in this study is a fully automated approach that is able to detect the luminal borders and identify and quantify the composition of the superficial plaque (depth: 1.5 mm). To our knowledge, this is the first approach that utilizes image processing, medical knowledge, and machine learning techniques to characterize different tissue types, and it is able to process without user interaction a sequence of good-quality FD-OCT images that do not have artifacts. The validation of the lumen detection methodology showed that our approach can address some of the common artifacts seen in FD-OCT images (i.e., the catheter artifact, small thrombi, or residual blood) and detect accurately the lumen borders (overlapping area ratio: 0.99 and nonoverlapping area ratio: 0.02). We also found good agreement between the manual annotations and the estimations of our methodology for the measured areas in the four plaque components (CA: $0.008\pm 1.96\times 0.04{\mathrm{mm}}^2$, LT: $0.06\pm 1.96\times 0.14{\mathrm{mm}}^2$, FT: $-0.09\pm 1.96\times 0.13{\mathrm{mm}}^2$, and MT: $-0.04\pm 1.96\times 0.1{\mathrm{mm}}^2$). In addition, the qualitative validation of our approach for the four tissue types demonstrated that our methodology characterizes correctly most of the pixels portraying these tissues (overlapping area ratio $>0.71$ for all tissue types). However, the nonoverlapping area analysis showed an increased nonoverlapping area ratio for the LT and MT (LT: 0.51 and MT: 0.47), indicating that our approach has a limited ability to detect the lateral extremities of these tissues. Additionally, there is a discrepancy between the approximation of the observers and our approach for the distal border of both tissues, which is not visible in OCT (Fig. 9).

To date, FD-OCT images were analyzed manually by expert observers. This process, however, is time consuming, it is restricted to a fraction of the obtained data, i.e., 1 frame at every 0.4- or 1-mm interval, and the accuracy of the reported results relied on the expertise of the observers. The proposed methodology appears to overcome the abovementioned limitations as it is able to process the FD-OCT data and provide reliable estimations for the lumen border and plaque components. However, further improvements are required and a more robust validation using histological data is needed before this methodology can have applications in a clinical setting. A potential combination of our methodology with the information provided by the data derived from the analysis of the reflected OCT signal (i.e., the backscattering and attenuation coefficients) is likely to provide additional information (i.e., detection of microstructures such as the presence of macrophages and neo-vessels) and allow more accurate detection of different tissue types and especially of the LT and MT.

7.

Conclusions

We present a fully automated methodology that is able to detect the lumen borders and characterize the composition of the superficial plaque in FD-OCT images. The methodology is based on the analysis of the FD-OCT images and was validated using the estimations of expert observers as gold standard. Our results demonstrated that it permits reliable border detection and characterization of the different tissue types with a typical morphology. Further validation of the proposed approach in tissues with typical morphology using histology data is needed before this methodology is used in the clinical and research arena.