1.

Introduction

Diagnosis of cancer at an early stage is important for effective management of the disease. Recently optical spectroscopy has received considerable attention for noninvasive, in situ, near-real-time diagnosis of cancer.^{1 2 3 4 5} For diagnosis, it exploits subtle changes in the spectra of tissue as tissue transforms from normal to malignant. Central to optical diagnosis is a diagnostic algorithm that can best extract the diagnostic features from the tissue spectra and accurately correlate them with the tissue histopathology. Most of the algorithms reported for optical diagnosis of cancer^{6 7 8 9 10 11 12 13 14 15 16 17 18} use traditional multivariate statistical techniques such as Fisher’s linear discriminant analysis,^{1 2 6 7 8 9} partial least-squares (PLS) analysis,¹⁰ singular value decomposition¹¹ (SVD), principal component analysis^{12 13 14 15} (PCA), etc. These classical linear techniques have the advantage of providing closed-form expressions that lead to simplicity in their design. However, they extract information from only the second-order correlation in the data and ignore higher order correlations, which could also be useful for improved discrimination.¹⁶ Use of nonlinear techniques¹⁶ is receiving attention for the purpose of development of algorithms since these could exploit higher order correlation. Artificial neural networks (ANNs) provide an array of nonlinear algorithms for feature extraction and classification^{16 17} and have also been used recently for laser-induced fluorescence (LIF) diagnosis of oral leukoplakia,¹⁸ cervical precancer,¹⁹ and atherosclerotic plaques²⁰ with excellent discrimination results. Van Staveren et al.¹⁸ demonstrated the use of multilayer ANN-based algorithms for autofluorescence detection of oral leukoplakia. The diagnostic algorithms based on ensembles of radial basis function (RBF) neural networks developed by Tumer et al.¹⁹ could identify cervical precancer more accurately when compared to their previous multivariate statistical algorithms. Rovithakis et al.²⁰ developed a higher order neural (HON)-network-based diagnostic algorithm and demonstrated its use for LIF detection of atherosclerotic plaques with excellent discrimination results. Apart from ANN-based algorithms, use of other state-of-the-art statistical pattern recognition techniques has also been reported recently.^{21 22} For example, Agrawal et al.²¹ used wavelet transforms and showed that features extracted from the polarized autofluorescence spectra of breast tissues through this transforms could serve as good discrimination indices. We recently showed that a nonlinear diagnostic algorithm based on the theory of maximum representation and discrimination feature (MRDF) can provide much improved diagnostic performance as compared to that based²² on linear PCA.

Another powerful recent approach for statistical pattern recognition based on machine learning is the theory of support vector machine (SVM), originally developed by Vapnik²³ and Burges.²⁴ SVMs have already received tremendous attention in a wide variety of classification problems^{24 25 26 27 28 29} and are being actively pursued for various theoretical extensions.^{30 31 32} The possibility of using SVMs for developing diagnostic algorithms is also attracting attention.^{33 34} While Palmer et al.³³ used a linear SVM classifier for classifying autofluorescence and diffuse reflectance spectra of breast tissues in vitro, Lin et al.³⁴ classified in vivo autofluorescence spectra from nasopharingeal tissues by using both the linear and the nonlinear SVM classifier with RBF kernel. In the reports of both the groups, the tissue spectra were dimensionally reduced by applying linear PCA algorithms prior to using the SVM approach for classification. Lin et al.³⁴ showed that the classification performance of an SVM classifier trained on the full spectral data was comparable to that obtained with the classifier trained on the diagnostically relevant principal components only. Their combined PCA-SVM approach was reported to have reduced computational complexity.

In this paper, we report, the use of an SVM for both feature extraction and classification jointly by integrating the approach of recursive feature elimination³⁵ (RFE) in the framework of an SVM (Refs. 23 and 24). RFE is a new technique developed recently by Guyon et al.³⁵ for extracting an optimal subset of nested features relevant for classification from a set of data with a vast number of features. Since RFE performs feature extraction using a performance criterion set by the classifier, the use of the integrated framework of SVM and RFE is expected to lead to a better classification performance compared to that with the use of an independent feature extractor such as PCA. We developed both linear and nonlinear SVM-based diagnostic algorithms using spectral data acquired in a clinical in vivo LIF study conducted on patients being screened for cancer of the oral cavity and normal volunteers. Although, in this paper, we focus on binary classification, i.e., cancerous versus normal, it can be easily extended to a multiclass classification using various approaches,³⁶ thereby enabling one to classify spectral data into more than two classes comprised of patients with various kinds of lesions of the oral cavity, for example, leukoplakia, erythroplakia, etc. in addition to cancerous and noncancerous lesions. In this paper, however, we focus on classifying spectral data of cancerous and normal tissue. We also compare the diagnostic efficacy of the SVM-based algorithms with that based on PCA and Fisher’s linear discriminant (FLD) using the same spectral data set. The algorithms based on SVM-RFE as well as SVM alone provide significantly improved diagnostic performance as compared to that based on both PCA and FLD in discriminating the cancerous tissue sites of the oral cancer patients from the healthy squamous tissue sites of normal volunteers as well as the uninvolved tissue sites of the patients with cancer of the oral cavity.

2.

Materials and Methods

In vivo autofluorescence spectra were recorded using a N ₂ -laser-(337-nm)-based portable fluorimeter reported earlier.^{15 22} It comprises a sealed-off pulsed N ₂ laser, a spectrograph (Acton Research Corporation, Acton, MA, USA), an optical fiber probe, and a gateable intensified CCD (ICCD) detector (4 Quik 05A, Stanford Computer Optics, Inc., Berkely, CA, USA). The spectral data acquisition was computer controlled. The autofluorescence spectra were recorded with the tip of the fiber optic probe placed in contact with the tissue surface. From each site, spectra were recorded in the 375- to 700-nm spectral range. During each measurement of tissue fluorescence, a reference spectrum was also acquired simultaneously from the phosphor-coated tip of an additional fiber illuminated with N ₂ laser radiation leaking from the other end of the N ₂ laser cavity. The peak of this reference spectrum was used to normalize the acquired tissue spectra and thus account for the observed pulse-to-pulse variation of the N ₂ laser power. The intensity of fluorescence from each tissue site is reported in this calibrated unit.

The study involved 13 normal volunteers with no history of the disease of the oral cavity and 16 patients selected from those enrolled for medical examination of the oral cavity at the outpatient department (OPD) of the Government Cancer Hospital, Indore. Informed consent was obtained from each patient as well as the normal volunteers who participated in this study. The patients included in this study had no history of malignancy and were suspected on visual examination by the concerned physician of having early cancer of the oral cavity. For these patients, biopsies were taken from the suspected areas subsequent to acquisition of spectra. Only those patients for whom histopathological diagnosis was squamous cell carcinoma (SCC), grade I, were included in this study. In vivo autofluorescence spectra were acquired from a total of 171 tissue sites from patients, of which 83 were SCC and the rest were uninvolved squamous tissue. Spectra were also recorded from 154 sites from healthy squamous tissue of normal volunteers. In each patient, the normal tissue sites interrogated were from the contralateral apparently uninvolved region of the oral cavity. On an average, five spectra from the cancerous tissue sites and four spectra from the uninvolved tissue sites were recorded. In normal volunteers, on an average, 10 spectra were recorded from the healthy squamous tissues. Each site was treated separately and classified via the diagnostic algorithm developed.

2.1.

Spectral Data

Each tissue fluorescence spectrum consisted of 717 intensity values (corresponding to 717 pixels of the ICCD) spanning the wavelength range of 375 to 700 nm. The autofluorescence spectra recorded from different cancerous and contralateral normal sites of the oral cavity of a patient are shown in Figs. 1(a) and 1(b), respectively. The considerable site-to-site variation in the spectra is apparent. The differences in the spectra from some of the normal and cancerous tissue sites are not that apparent, because they are masked by the large intrapatient and interpatient variability in the intensity and line shapes. While some of this variation may represent intrinsic variation in tissue fluorescence, the variable nature of the contact of the probe with the tissue surface in a clinical situation will also add to the variation. It is pertinent to note that in the in vitro studies on oral cavity tissues,³⁷ where the variability due to the nature of contact of probe with tissue surface is expected to be minimal, a percentage variation (σ/x¯) of ∼30% was observed in the spectrally integrated intensities (∑ I) from different sites of normal or cancerous tissues over the total sample size investigated. Here, x¯ is the mean of ∑ I values from different sites of one category and σ is the standard deviation. In comparison, in this in vivo study, the percentage variation (σ/x¯) in (∑ I) was ∼60%. To ensure good discrimination, it is necessary to minimize these variations that may obscure the intercategory differences. To minimize the inter- and intrapatient variability, a two-step procedure for preprocessing of the raw spectral data was adopted. In the first step, the mean spectrum over all the healthy squamous tissue sites of the normal volunteers was calculated and subtracted from the spectrum of each tissue site of the oral cavity of patients as well as of normal volunteers. Since mean subtraction displays the differences in the spectra of the diseased with respect to the mean spectra of the healthy squamous tissue, it is expected to lead to enhancement of spectral differences between the two diagnostic categories. Next, the resultant spectrum of each category was normalized with respect to the standard deviation of the spectra of that category. This normalization is expected to remove from the spectra the influence of scatter in the spectral intensity by making the standard deviation of the spectra of each diagnostic category equal to unity. Indeed, mean subtraction followed by normalization of the spectra with respect to their respective standard deviations made the spectral differences between the two diagnostic categories much more apparent. Figure 2 shows the spectra for cancerous and uninvolved sites of the oral cavity of the same patient after preprocessing. Note here that the differences in the preprocessed spectra from cancerous and contralateral uninvolved tissue sites of the same patient are generally more distinct^{12 37} as compared to the differences when preprocessed spectra from similar tissue sites of all the patients are considered as a whole. Various earlier reports on measurements of tissue fluorophores³⁸ as well as tissue parameters³⁹ also demonstrate this effect. Figure 3 shows the preprocessed spectra from cancerous and contralateral normal tissue sites of four patients chosen at random. It is evident from the figure that the interpatient differences in the preprocessed spectra do not appear to be that prominent in comparison with the intrapatient differences shown in Fig. 2.

Figure 1

N ₂ -laser-excited autofluorescence spectra recorded from (a) squamous cell carcinoma tissue sites (solid line) and (b) uninvolved tissue sites (dashed line) of the same patient.

Figure 2

Preprocessed autofluorescence spectra from squamous cell carcinoma tissue sites (solid line) and from uninvolved squamous tissue sites (dashed line) of the oral cavity of the same patient.

Figure 3

Preprocessed autofluorescence spectra from squamous cell carcinoma tissue sites (solid line) and from uninvolved squamous tissue sites (dashed line) of the oral cavity of four patients chosen at random.

3.

Results

Table 1 lists the diagnostic results obtained with a linear SVM classifier trained on the spectral data set corresponding to raw spectra and preprocessed spectra with the full set of spectral features (i.e., N=717 intensity values). For comparison’s sake, the classification results yielded by a conventional NMC on the same data sets are also listed in the same table. An NMC is based on least Euclidean distance of the test features from the means of the prototype features of the corresponding tissue types in the training set. The sensitivity and specificity values for the training set data were obtained on the basis of leave-one-out cross-validation. It is evident from the table that the sensitivity and specificity values in the training and the validation data sets are much improved with the preprocessed spectral data as compared to the unprocessed raw spectral data. Therefore, we extended the subsequent exercise on algorithm development only with the preprocessed spectral data sets.

Table 1

Figures 4 and 5 demonstrate the leave-one-out cross-validation error as a function of the degrees (d) of the polynomial kernel and the widths (σ) of the Gaussian RBF kernel, respectively. From Fig. 4 it is clear that the leave-one-out error is the minimum for d=2, and therefore, we used polynomial kernel of degree 2 for training the polynomial SVM classifier for algorithm development. Figure 5 shows that leave-one-out error is the minimum at more than one σ value (e.g., at σ=50, 75, and 100). However, for the σ value of 100 the total number of cancerous samples misclassified was the minimum and therefore, σ=100 was used as the width of the RBF kernel for subsequent training of the RBF SVM classifier.

Figure 4

Leave-one-out cross-validation error in the training set data as a function of the degree of the polynomial kernel for the polynomial SVM classifier.

Figure 5

Leave-one-out cross validation error in the training set data as a function of the width of the Gaussian radial basis function for the RBF SVM classifier.

To train an SVM algorithm one must supply a priori the value of the regularization parameter C to the learning algorithm. Since no established guideline exists in the SVM methodology^{23 24} as to what should be the optimal value of C, the linear and nonlinear SVM classifiers were trained with different values of C (C=1, 10, and ∞). It was found that the classifier with C=∞ gave the best generalized classification performance, i.e., the total misclassification error over the training (leave-one-out cross-validation) and the independent validation data sets was the least. Therefore, for subsequent feature subset selection with the RFE algorithm we trained each of the SVM classifiers with C=∞ at each iteration stage. To evaluate the diagnostic contribution of each selected subset of features at each iteration stage of the RFE algorithm, we set the cost function as the total number of samples misclassified by the classifier in the independent validation set as well as in the training set data with leave-one-out cross-validation. The optimal subset of features was the one for which the total number of misclassified samples was the minimum. If the total number of misclassified samples was the same for more than one feature subsets, then the feature subset for which the total number of cancerous samples misclassified was minimum, was selected.

The total misclassification error for the 11 nested subsets of features was determined with the SVM RFE method. The results for linear, polynomial, and RBF kernels are shown in Figs. 6 to 7 8, respectively. It is evident from the figures that while the misclassification error is the minimum for the linear SVM classifier trained with the subset of 45 features ranked by the SVM RFE algorithm, for both the polynomial SVM and the RBF SVM the respective misclassification errors are the minimum with the subset of 350 features ranked by the respective SVM RFE algorithms. The diagnostic performances of the SVM RFE algorithms with linear, polynomial, and RBF kernels are listed in Tables 2 to 3 4. The sensitivity and specificity values for the training set data represent the leave-one-out cross-validation values.

Figure 6

Total misclassification error in the training and the validation data set as a function of the 11 nested subsets of features obtained using SVM RFE algorithm with a linear kernel.

Figure 7

Total misclassification error in the training and the validation data set as a function of the 11 nested subsets of features obtained using the SVM RFE algorithm with a polynomial kernel.

Figure 8

Total misclassification error in the training and the validation data set as a function of the 11 nested subsets of features obtained using the SVM RFE algorithm with an RBF kernel.

Table 2

Table 3

Table 4

Similarly, for the development of the RFE algorithm with the FLD classifier, it was trained on the training set data and tested on the training set (leave-one-out cross-validation) as well as on both the independent validation data sets at each iteration stage. Figure 9 shows the total misclassification error as a function of the 11 nested subsets of features obtained with the FLD RFE method. The figure clearly shows that the misclassification error is the minimum with the subset of 45 features ranked by the FLD RFE algorithm. Table 5 summarizes the diagnostic results obtained with the FLD RFE algorithm. Here also, the sensitivity and specificity values for the training set data represent the leave-one-out cross-validation values.

Figure 9

Total misclassification error in the training and the validation data set as a function of the 11 nested subsets of features selected through RFE for FLD classifier.

Table 5

Table 6 lists the sensitivity and specificity values for the training and the validation data sets obtained using the FLD as well as the three SVM algorithms with the full spectral features as well as with the optimal subset of features selected using the respective RFE algorithms. For comparison’s sake, the sensitivity and specificity values obtained using the linear PCA-based algorithm as well as the linear SVM algorithm trained with the diagnostically relevant principal components (PCs) are listed in Table 7. PCA of preprocessed spectra resulted in six PCs that collectively accounted for 99.5% of the total variance of the spectral data. Of the six PCs, only four (PC 1, PC 3, PC 4, and PC 5) were found to have significantly different (p<0.001) values for SCC and normal squamous tissue. Therefore, these four PCs, which together accounted for 79% of the total variance (PC 1 accounting for 70%, PC 3 for 6%, and PC 4 for 2% of the total variance, and PC 5 accounting the remaining 1%) were used for subsequent classification.

Table 6

Table 7

Figure 10 shows the ROC curves generated for the different diagnostic tests based on SVMs and FLD. To quantify the accuracy of the tests, the areas under the curves were also estimated. Table 8 lists the area under the curve values for the ROC curves corresponding to linear, polynomial, and RBF SVM diagnostic algorithms.

Figure 10

ROC curves for different diagnostic algorithms based on SVMs (linear, polynomial, and RBF) and FLD.

Table 8

4.

Discussion

In Table 1, we summarized the diagnostic performance of the SVM classifier and the nearest mean classifier. For both the classifiers, the diagnostic results were obtained using the preprocessed spectral data and the unprocessed raw spectral data. The results clearly indicate that SVM outperforms the NMC for both data sets. The superior classification performance of the SVM classifier originates from the built-in capability of the SVM approach to separate classes that are not linearly separable in the original parametric space.²⁴ The advantage of the two-step preprocessing of the raw spectral data, as described in the previous sections, is also apparent from the table.

The diagnostic performances of the SVM-based and the linear-PCA-based algorithms over the training and the two independent validation data sets are listed in Table 7. For SVM-based algorithms, classification results were obtained for two cases. In one case, SVM was used as a classifier with PCA, providing the diagnostically relevant features (SVM PCA), and in the second case, SVM was used for classification with the full set of spectral features as well as for both feature extraction and classification jointly using the SVM-RFE approach. A perusal of the table shows that the SVM-based algorithms have resulted in significantly improved classification performance as compared to that obtained with the PCA-based algorithms. Further, in view of the previous work by Lin et al.³⁴ using the SVM PCA, our results show that the integrated SVM-RFE approach gives considerably improved diagnostic performance as compared to the SVM-PCA algorithm developed based on the same data set. This is not surprising because PCA, which is basically an independent feature extractor,¹⁶ extracts features by projecting the input data into a new feature space of lower dimensionality through a linear transformation matrix. PCA optimizes the transformation matrix by finding the largest variations in the original input space,^{16 42} thereby reducing the dimension of the original data by optimally representing the data in the form of a few PCs (which are linear combinations of the original data). However, the PCs do not ensure any class-discriminatory information. The drawback of an independent feature extraction algorithm such as PCA is that its optimization criterion is different from the classifier’s minimum classification error criterion,¹⁶ which can cause inconsistency between feature extraction and classification stages of a diagnostic algorithm and, consequently, may degrade the performance of classifiers. This problem is overcome by pursuing the integrated approach of SVM and RFE, since RFE performs feature extraction by selecting the diagnostically relevant input variables while using the performance criterion set by the classifier itself.³⁵ Further, note also that computational complexity is also not reduced in the SVM-PCA approach, because the SVM classification operation does not depend on the dimensionality of the feature space, which can be even infinite.²⁴ Perhaps the SVM-PCA approach could be little faster, but at the cost of classification performance. However, the speed aspect should not matter when developing an off-line diagnostic algorithm where the main focus is the accuracy and simplicity. Speed requirements can also be taken care of by the SVM-RFE approach, which practically performs dimension reduction through feature selection.

In Table 6, we list the diagnostic performance of the FLD and the three SVM algorithms with the full spectral features as well as with the optimal subset of features selected using the respective RFE algorithms. A perusal of the table shows that SVM-based algorithms provide significantly improved diagnostic performance as compared to FLD. While the difference in diagnostic performance is particularly large for full set of spectral features, it is reduced for an optimal subset of features (selected by the respective RFE algorithms). The ROC analysis of the classification results provides a more critical evaluation. Figure 10 shows that while all three ROC curves corresponding to the SVM-based algorithms are very close to the point of ideal performance (i.e., the upper left-hand corner), the ROC curve corresponding to the FLD-based algorithm is quite far away from the ideal point. This is further supported by the observations of significantly higher values of the area under the ROC curves (Table 8) corresponding to the SVM-based algorithms as compared to that based on FLD with the performance of the RBF-SVM algorithm being the best.

The large improvement in diagnostic performance of SVM-based algorithms as compared to that based on classical FLD, appears to be due to the fact that while FLD extracts information from only the second-order correlations in the input spectral data⁴² (covariance matrix) to enhance the class-discriminatory information, the SVMs use higher order correlations.²⁴ Note also that FLD optimizes the transformation matrix by finding the largest ratio of between-class variation and within-class variation when projecting the original input data to a feature space of lower dimension.⁴² Thus, it considers the squared separation between the means of each class and, therefore, is not expected to perform well on nonsymmetric data such as the LIF spectral data that may have multiple clusters per class.⁴⁵ This follows because when input data has multiple clusters per class it might so happen that the mean for a class of two clusters can lie close to the mean of another class.

From the viewpoint of pattern recognition, the task of tissue classification based on LIF spectral data is a pattern classification problem, and the feature vector for classification comprises the measured intensities corresponding to the different pixels (of the detector) that specify the dimension of the features. If working directly with all these spectral features whose dimension is much higher (N=717 in this case) as compared to the size (∼119 in this case) of the training samples, the classifier might suffer from the so-called “curse of dimensionality,”¹⁶ causing it to have poor generalization in classification performance. The use of RFE not only helps in choosing an optimal subset of features that are relevant for classification, but also reduces the feature dimension by solving the “curse of dimensionality” problem. This is evident from the observed large improvements in the classification performance of the diagnostic algorithm based on FLD for the optimal subset as compared to the full set of spectral features, leading to an increase of 29.5 and 49.5%, respectively, in the average sensitivity and specificity values. Note, however, that the diagnostic algorithms based on SVM are not too sensitive to the selection of optimal subset of features. For example, while for the linear SVM, the average sensitivity and specificity values improve by 6 and 4%, respectively, by going from the full set to the optimal subset of spectral features, no change in sensitivity and only a 2% increase in specificity values was observed for the polynomial SVM. For the RBF SVM, the resulting improvements in the average sensitivity and specificity values were found to be by 1 and 3.5%, respectively. This highlights the built-in capability of the SVM to sidestep overfitting to a large extent, despite the fact that it was trained on a set of training data where the number of features is large compared to the size of training patterns²⁴ (e.g., the present set of LIF spectral data). Further, an added advantage of using RFE is that it directly becomes clear which spectral regions dominate the classification problem, in contrast to using PCA for dimension reduction, where one must perform a component-loading operation to get back the spectral regions of interest. For example, the optimal subset of 350 features selected by the SVM RFE with RBF kernel correspond to wavelengths that span nearly the entire spectral region going from 355 to 700 nm.

Also note here that although the standard SVMs are designed for binary classification,^{23 24} multiclass classification problems could be solved either by directly constructing a multiclass SVM classifier³⁶ or by using voting scheme methods based on combining many basic binary SVM decision functions.³⁶ For example, one-against-all decomposition is the most commonly used voting scheme method. In this case, the classification problem to k classes is decomposed to k binary SVM decision functions f_m(x), m∈K={1,…,k}, where the decision function f_m(x) separates training data of the m’th class from the training patterns of other classes. The classification of a pattern x is performed according to maximal value of the functions f_m(x), i.e., the label of x is computed as arg max _m∈K f_m(x). The development SVM algorithms for multiclass classification is underway in our group. Here, the potential of the SVM to simultaneously classify spectral data into more than two classes comprising patients with various kinds of lesions of the oral cavity, for example, leukoplakia, erythroplakia, etc., in addition to cancerous and noncancerous tissues will be explored.

Note also here that the development of diagnostic algorithms described here was based on spectral data from patients who belonged to a high-risk population (were suspected of having SCC on visual examination). This patient selection criteria might influence the sensitivity and specificity values obtained in this study. However, the motivation for this work was to compare the relative performance of the different types of diagnostic algorithms using the same spectral data set from the same population of patients. The patient selection criterion is unlikely to influence this comparison.

5.

Conclusions

The application of an integrated framework of the SVM and RFE for discrimination of early squamous cell carcinoma from the normal squamous tissue sites of the oral cavity was reported. The flexibility of the framework of the SVM-RFE algorithm makes it convenient to conduct feature extraction and classification jointly, leading to improved classification results. Both linear- and nonlinear-SVM-based diagnostic algorithms were developed using spectral data acquired in a clinical in vivo LIF study conducted on patients being screened for cancer of the oral cavity and normal volunteers. The relative diagnostic performances of the algorithms have been evaluated and also compared with that of the classical FLD and PCA-based algorithms. The results show significantly improved classification performance of the integrated SVM-RFE algorithms as compared to both FLD and PCA-based algorithms.