2.1.

Experimental System

A schematic of the OPRS system is shown in Fig. 1(a). An illumination fiber ($100\text{-}\mu \mathrm{m}$ core diameter, $\mathrm{NA}=0.12$) delivered light to tissue from a 20-W tungsten halogen broadband light source (Ocean Optics, HL2000HP-FHSA). Then, light scattered from the tissue was collected by detection fibers ($100\text{-}\mu \mathrm{m}$ core diameter, $\mathrm{NA}=0.12$) and was delivered to an imaging spectrometer (PI Acton SpectraPRO SP-2356, Pixis 2KB) equipped with a $150\mathrm{g}/\mathrm{mm}$ grating optimized for the visible wavelength region (500-nm blaze). To enable a modular design wherein multiple probes could be easily interchanged, proximal ends of illumination and detection fibers were connected using low insertion loss (ca. 0.15 dB) FC/APC adapters to coupling fibers leading to either the light source (the illumination fiber) or the imaging spectrometer (detection fibers). At the spectrometer, detection fibers were assembled in a vertical array using a slit ferrule for alignment with the spectrometer’s entrance slit. Light from all collection fibers was simultaneously dispersed by the spectrometer’s grating onto an imaging CCD. An image was produced with the vertical dimension corresponding to different detection fibers and the horizontal axis displaying scattering spectra collected by the fibers thus simultaneously capturing spectra from all detection fibers in a single image. The entire system was housed on a wheeled cart for mobility [Fig. 1(b)].

Fig. 1

(a) Schematic of the overall system for OPRS. (b) Picture of the clinical system on a wheeled cart. (c) Illustration of the distal end of the OPRS probe: I denotes the illumination fiber; BF labels individual detection fibers where BF1, BF2, and BF3 collected copolarized (parallel) component of tissue scattering and BF1per, BF2per, and BF3per collected cross-polarized (perpendicular) scattering. (d) Artistic three-dimensional rendering and (e) an actual image of the distal end of OPRS after fiber polishing (scale bar: 1 mm). (f) Orientation of polarization transmission axes of polarizing film relative to the fiber array of the OPRS probe.

A schematic of the distal end of the probe for OPRS is shown in Fig. 1(c). All fibers had a silica core with $n=1.458$ and diameter of $100\mu \mathrm{m}$; fluorine-doped cladding with $n=1.455$ and diameter—$110\mu \mathrm{m}$; and an NA of 0.12 (CeramOptec Industries, Inc., WF 100/110 P12). Obliquely oriented collection fibers are referenced according to their distance from the illumination fiber—BF1 is separated from the illumination fiber by one flat spacer fiber; BF2 is separated from the illumination fiber by two fibers, etc. Two flat tip fibers were positioned on each side of the illumination fiber as spacers to accommodate the gap between two pieces of the polarizing film that was used for polarized light illumination/detection as described below. The detection fibers (BFs) were polished at 40 deg with respect to the sample surface using a custom-made polishing puck. We have reported previously that this angle provides the optimum combination of depth selectivity and collection efficiency.⁴⁵ In this multifiber design, collection cones of detection fibers and the illumination beam overlap at progressively increasing depth in tissue as the distance between the illumination and detection fibers increases thus allowing control over the sampling depth of tissue optical properties. The diameter of the fiber array of the constructed probe was ca. 1.6 mm. The outer probe diameter was made 7.6 mm for handling convenience in applications for detection of precancers in the oral cavity. An artistic rendering and an image of the distal end of the OPRS probe is shown in Figs. 1(d) and 1(e), respectively, to illustrate the beveled probe design following angle polishing. Once the fibers were polished, two pieces of $150\text{-}\mu \mathrm{m}$ thick polarizing film ($n=1.458$) with 0.0002% extinction transmittance of cross-polarized light were glued to the distal end of the fibers using optically transparent and biocompatible epoxy (Epo-Tek 301-2). The two polarizing film pieces were positioned with their polarization axis perpendicular to each other as shown in Fig. 1(f), which allowed one half of the detection fibers to collect light copolarized with the illumination light polarization and the other half to collect cross-polarized light. Here, the scattered light collected with polarization parallel to the illumination is termed parallel and the cross-polarized light is defined as perpendicular. The fiber assembly with polarizing films was secured inside steel tubing with the outer diameter of 7.6 mm using biocompatible epoxy glue (Epo-Tek 301-2). Then, the steel tubing was carefully bent at an angle of 45 deg for easy positioning inside the oral cavity. Finally, a $200\text{-}\mu \mathrm{m}$ thick quartz window ($n=1.54$) was glued to the polarizing film for protection of the probe during sterilization and to achieve an optimum overlap between illumination and collection cones at tissue surface.

2.2.

Collection of Clinical Data

In vivo spectra were collected with informed consent from 28 patients who were 18 years old and over and who were referred to the Department of Integrative Oncology at the British Columbia Cancer Agency (BCCA) with lesions in the oral cavity suspicious for dysplasia or carcinoma. The spectroscopic measurements followed standard oral cavity examinations by a physician. The spectra were collected from all sites suspicious for dysplasia and from contralateral (whenever possible) normal sites. The abnormal sites were biopsied and were histologically confirmed as benign, mild dysplasia (MD), or severe dysplasia (SD). Benign sites appeared abnormal during clinical examination but were histologically diagnosed as normal. Several (2 to 3) normal site measurements were taken for each patient. Depending on the size of an abnormal lesion, one to several measurements per lesion were obtained. All measurements were taken with room lights turned off to minimize background signal. Calibration spectra were acquired before each patient evaluation using a diffuse reflectance substrate standard (SRS-99, Labsphere, Inc.) and a background signal was measured using minimally reflecting black substrate (SRS-02, Labsphere, Inc.). Measurements from three patients were removed due to errors during data collection process associated with user mishandling of the probe or system malfunctioning.

2.3.

Preprocessing of Oblique Polarized Reflectance Spectroscopy Spectra

The mathematical equation that was used in spectra preprocessing is given below and it includes the following steps. First, the background signal from the minimally reflecting black substrate ($I_{\text{dark}}$) was subtracted from the collected raw spectra ($I_{\mathrm{meas}}$) to remove residual environmental light. Background signal from the environment was minimized by carrying out measurements in a dark room. Next, the spectral responses of the source, fibers, and detector were accounted for by normalizing the background corrected spectra by the spectrum from the diffuse reflectance substrate standard ($I_{\mathrm{whte}}$). Because the diffuse reflectance standard is not a perfect depolarizer,⁴⁶ the perpendicular signal of the reflectance standard was multiplied by a ratio of the parallel to the perpendicular component called the depolarization coefficient ($D$) to account for this effect. To minimize background from residual room light, background signals ($I_{\text{dark}}$) were first subtracted from the parallel and perpendicular spectra. In addition, collection areas ($A$) at the tissue interface for each detection fiber was determined in Zemax (Zemax, LLC, Kirkland, Washington) and was used to correct for the trend that fibers farther from the illumination fiber collect scattered photons from larger areas. To correct for the variations in the collection efficiency of detection fibers, the spectra were divided by the power throughput of each fiber ($P$). The relative power throughputs of detection fibers ($P_{\mathrm{par}}$ and $P_{\mathrm{per}}$) were determined by connecting each collection fiber’s promixal end to the illumination light source and measuring the power at the fiber’s distal end with a power meter (371R Optical Power Meter, Graseby Optronics). Also, differences in the collection time ($t$) that was used during data acquisition were accounted for. In summary, the normalization scheme to produce comparable OPRS data was as

2.4.

Quantifying Penetration Depth of Collection Fibers

To quantify the depth in a turbid media from which collection fibers collect scattering signal, an experiment was adapted from a method developed in Refs. 38 and 47. Briefly, a glass container with a 2-cm thick layer of optically transparent cured polydimethylsiloxane (PDMS) at the bottom to eliminate any back-reflections was filled with 20% Intralipid (Sigma-Aldrich, I141) to mimic stromal scattering (${\mu }_s^{\prime }=2{\mathrm{mm}}^{-1}$)^15,48 at 600 nm. This wavelength was used as it was the central wavelength in our range of interest (450 to 750 nm). The concentration of Intralipid in water required to simulate the target scattering properties was determined by measuring transmission of phantoms with varying Intralipid concentrations. Then, using the relationship ${\mu }_s=\frac{-\ln (T)}{L}$, where $L$ is the path length, a plot of Intralipid scattering properties as a function of Intralipid concentration was obtained (see Appendix). During measurements, the OPRS probe was initially placed in contact with the transparent PDMS layer. Then, the probe was moved away from the PDMS at $50\text{-}\mu \mathrm{m}$ increments until a distance of $2500\mu \mathrm{m}$ was reached. OPRS measurements were carried out at each increment. Spectra from each collection fiber were processed as described in Sec. 2.3 and integrated in the wavelength range 450 to 750 nm. The integrated intensities were plotted as a function of Intralipid thickness.

2.5.

Standardization of Clinical Oblique Polarized Reflectance Spectroscopy Measurements

Standardization with respect to the normal tissue was carried out to account for the varying anatomy across patients. The standardization approach was adapted from a method by Rajaram et al.⁴⁹ wherein it was applied to account for intersubject variations in the skin. A standardization factor was used to account for divergence of scattering from normal sites obtained from a given patient from the overall average intensity of all collected normal spectra. The standardization factor for each patient ($S_i$) was calculated by dividing the integrated intensity of the average spectrum of all collected normal spectra ($N_{\text{mean}}$) by the integrated intensity of a normal spectrum from each patient ($N_i$): $S_i=N_{\text{mean}}/N_i$. The $S_i$ factor for each patient was used to normalize intensity of all measured normal and abnormal spectra as follows: $M_i{(\lambda )}^{\prime }=S_i\times M_i(\lambda )$, where $M_i{(\lambda )}^{\prime }$ and $M_i(\lambda )$ represent the standardized and originally measured spectra, respectively. Note that $M_i(\lambda )$ represents any measured $I_i{(\lambda )}_{\mathrm{par}}$ and $I_i{(\lambda )}_{\mathrm{per}}$ spectra. Figure 2 shows OPRS spectra before and after standardization. The standardization scheme retained differences in the spectral shape of the measurements from the normal and abnormal lesions while removing large variations in the overall magnitude caused by interpatient variations and by various anatomical locations in the oral cavity.

Fig. 2

OPRS spectra pre- and poststandardization.

2.6.

Spectral features

The following eight binary classification tasks were evaluated: (1) normal versus MD, (2) normal versus SD, (3) normal versus MD and SD combined, (4) MD versus SD, (5) benign versus MD, (6) benign versus SD, (7) benign versus MD and SD combined, and (8) benign versus normal. The list of all OPRS features considered in data analysis is shown in Table 1. It includes the spectral mean, which is the average intensity taken across the entire wavelength range (450 to 750 nm), along with the intensity at the most discriminatory wavelengths that were extracted for the following spectra: parallel ($\parallel$)—collected by fibers BF1, BF2, and BF3; perpendicular ($\perp$)—fibers $\mathrm{BF}{1}_{\mathrm{per}}$, $\mathrm{BF}{2}_{\mathrm{per}}$, and $\mathrm{BF}{3}_{\mathrm{per}}$; diffuse ($\parallel +\perp$)—sums of signals collected by symmetrically positioned fibers collecting parallel and perpendicular spectra; polarization gated ($\parallel -\perp$)—differences between parallel signals and corresponding perpendicular signals; parallel/perpendicular ($\parallel /\perp$)—ratios of parallel and perpendicular signals collected by symmetrically positioned fibers; parallel and perpendicular differentials—differences of signals collected by two adjacent fibers; ratios of differential signals; and ratio of polarization gated signals. All these features were extracted from preprocessed and standardized spectra. In addition, the spectra were normalized by the area under the curve (AUC) to emphasize spectral shape differences between various diagnostic categories. For each binary classification task, the most discriminatory wavelength was selected as the wavelength with the highest Welch’s $t$-statistic value. The Welch’s $t$-statistic at a given wavelength calculates the absolute difference in mean spectra across patients between the two diagnostic classes relative to the amount of interpatient spectral variation that is observed within the two classes at that wavelength. Another feature that was extracted for all detection fibers was the ratio of the intensities at 576 to 610 nm; this ratio reflects the magnitude of hemoglobin absorption in the scattering spectra.⁴⁵

Table 1

Individual features that were considered for analysis.

2.7.

Selecting the Most Discriminatory Spectral Features

A total of 120 spectral features were extracted from measured OPRS spectra. These features include 27 spectral means, 27 intensities at the most discriminatory wavelengths, and 6 intensity ratios at $576/610\mathrm{nm}$ for the total of 60 features for each unnormalized and normalized spectra with the final count of 120 features; this translates to ($2^{120}-1$) different possible feature combinations in a two-class classification problem. An exhaustive investigation of all feature combinations would require vast amounts of processing time, and the available data sample size is likely to be insufficient for investigating high-dimensional feature spaces. Therefore, it was necessary to reduce the number of features prior to training classifiers for differentiating between different diagnostic classes. Feature selection was used to identify the most diagnostically relevant features and to reduce redundancies by eliminating features that are closely related to each other. We preferred feature selection over feature extraction (such as principal component analysis) to retain the physical significance of features used for diagnostic classification. Maximum relevance minimum redundancy (mRMR) was employed as the approach for feature selection as mRMR produces a subset of features with the highest relevance (i.e., highest discrimination between the two diagnostic classes) and minimal redundancy (i.e., minimum correlation between features). mRMR has low computational complexity and produces features with smaller classification errors as compared to those obtained from other feature selection strategies.⁵⁰ Given that sample size is limited for some of the diagnostic classification tasks, leave-one-out cross validation (LOOCV) strategy was used to reduce overtraining. The appropriate set of features from mRMR was calculated as the minimum feature set beyond which the performance on training data does not significantly improve by inclusion of additional features. Upon selecting the most diagnostically relevant features, their performance in discriminating between different diagnostic classes was evaluated by determining their classification accuracy. Random forest classifiers were used in this study owing to their advantages when dealing with small sample sizes and high-dimensional feature space.⁵⁰ The area under the nonparametric receiver-operating characteristic (ROC) curves (AUC) was calculated to quantify the performance of selected features when combined by the random forest classifier. The random forest classifications along with the determination of the discriminatory wavelengths and feature selection were performed in the R software using the caret and mRMRe packages (R Development Core Team, Vienna, Austria).

2.8.

Testing for Overtraining

To reduce the risk of overtraining associated with the limited number of data in the clinical trial, we employed LOOCV, which leaves out a single subject to generate the random forest models. To further test for overtraining, a permutation test was employed wherein the pathology definition (normal, benign, etc.) was randomly rearranged while preserving the number of cases in each diagnostic category.⁵¹ The mean and standard error of the AUCs were obtained from shuffling the pathology definitions 100 times. These values were then compared with the AUC values obtained with correctly assigned diagnostic results to the data.

3.1.

Depth Penetration of Oblique Polarized Reflectance Spectroscopy Collection Fibers

Penetration depths of detection fibers of the OPRS probe were evaluated using an Intralipid phantom simulating stromal scattering. The relationship ${\mu }_s=\frac{-\ln (T)}{L}$ between the reduced scattering coefficient, ${\mu }_{\mathrm{s}}^{\prime }$ ($g=0.752$)⁵² at 600 nm, and transmission of Intralipid phantoms at concentrations of 0.05%, 0.1, 0.15%, 0.2%, and 0.25% was used to plot the dependence of the scattering coefficient on the Intralipid concentration (see Appendix). The fit was found to be in good agreement with previously published results.^52,53 Using the derived linear fit, the Intralipid concentration of 2.6% in water was found to yield the target ${\mu }_s^{\prime }$ of $2{\mathrm{mm}}^{-1}$. Integrated intensities of scattering signals collected by detection fibers were plotted as a function of phantom thickness as shown in Fig. 3. The signals followed a common trend with an initial increase followed by a saturation. As expected, the intensity profiles of detection fibers positioned further away from the illumination fiber were shifted toward greater depths [Fig. 3(b)]. Also, the perpendicular signals appeared at greater depths as compared to the parallel ones as the result of a gradual depolarization of linear polarized excitation with depth; this trend became less pronounced with increased separation between illumination and detection fibers. The detected signals achieved 90% value of the saturated signal at 750, 900, and $1100\mu \mathrm{m}$ for parallel detection fibers BF1, BF2, and BF3, respectively, and at 1200, 1250, and $1400\mu \mathrm{m}$ for perpendicular fibers BF1per, BF2per, and BF3per, respectively [Fig. 3(c)]. The polarization gating (BF1 − BF1per, BF1 − BF2per, and BF3 − BF3per) reduced the interrogation depth in the scattering medium as compared to individual collection fibers [Fig. 3(c)].

Fig. 3

(a) A schematic of experimental setup to quantify depth penetration of OPRS collection fibers. (b) Integrated intensities of scattered signal collected by parallel (solid circle) and perpendicular (dotted square) detection fibers as a function of thickness of Intralipid phantom simulating stromal scattering (${\mu }_s^{\prime }=2{\mathrm{mm}}^{-1}$). (c) Depths at 90% signal saturation for individual collection fibers and polarization gated signals.

3.2.

Sample Distribution of Clinical Measurements

Results of 93 in situ measurements from 25 patients were analyzed. The distribution of anatomical sites measured is shown in Table 2. Representative histological examples of different diagnostic categories, as shown in Fig. 4, illustrate morphological changes associated with the progression of dysplasia in the oral cavity. It is important to note a high degree of similarity between anatomy of benign and MD categories. MD is marked by early dysplastic changes limited to the lower third of the epithelium in the basal and parabasal layers that are not present in the benign category. However, both the benign and the MD images show a well-defined superficial keratinization layer that complicates diagnosis.

Table 2

Distribution of anatomical sites.

Fig. 4

Representative images of biopsied tissue sites confirmed as normal, benign, MD, and SD. (scale bar: $200\mu \mathrm{m}$). Tissue slices were stained with H&E for standard histopathological analysis.

To evaluate the prevalence of keratinization, mean thicknesses of the epithelium and keratin layers for benign, MD, and SD diagnostic categories were measured in scanned H&E stained slides of biopsied samples using Panoramic Viewer (3DHISTECH) (Fig. 5). Mean epithelial thicknesses (epithelium + keratin) for benign, MD, and SD were ca. 426, 504, and $701\mu \mathrm{m}$, respectively, indicating thickening of the epithelial layer with dysplasia progression. Lesions in the oral cavity often experience hyperkeratosis due to chronic irritation, which is reflected in an increased degree of keratinization.⁵⁴

Fig. 5

Mean epithelial and keratin thicknesses of biopsied sites (benign, MD, and SD).

3.3.

Polarized Reflectance Spectra

Averaged parallel ($\parallel$), perpendicular ($\perp$), diffuse ($\parallel +\perp$), and polarization gated ($\parallel -\perp$) reflectance spectra for each diagnostic category are shown in Fig. 6(b). Qualitative analyses of the spectra uncovered substantial differences between diagnostic categories. These differences were wavelength dependent and were quite substantial in certain narrow wavelength regions that prompted analysis of the most discriminatory wavelengths and wavelength regions that are described below. Overall, average benign spectra tended to have the highest intensity as compared to other diagnostic categories likely due to a high degree of keratinization associated with benign oral cavity lesions. The mean abnormal spectra (MD and SD) exhibited larger intensities than the normal spectra that can be attributed to higher scattering due to keratinization of these sites and to morphological changes associated with dysplasia, such as increased nuclear size, hyperchromasia, and pleomorphism.⁵⁵ The spectra were also normalized by the AUC to emphasize shape differences between diagnostic categories (Fig. 7). It is interesting to note that the normalization revealed small but distinct spectral differences between all diagnostic categories in the parallel spectra; however, in the perpendicular spectra, MD and SD categories are virtually indistinguishable, whereas normal and benign spectra are clearly separated.

Fig. 6

Averaged spectra for each diagnostic category for parallel ($\parallel$), perpendicular ($\perp$), diffuse ($\parallel +\perp$), and polarization gated ($\parallel -\perp$) reflectance spectra.

Fig. 7

Polarized reflectance spectra normalized to the AUC; all collected spectra for each diagnostic category were first normalized by the AUC and then averaged.

3.4.

Diagnostically Significant Spectral Features

Figure 8 shows a diagram that summarizes the most discriminatory wavelengths that provide the maximum separation between each binary diagnostic classification class. Each tickmark on the $x$-axis represents a spectral feature of interest listed in Table 1—a total of 27 features for each binary classification; the corresponding discriminatory wavelengths are identified by dots along the $y$-axis. The analysis was carried out for unnormalized [Fig. 8(a)] and AUC normalized spectra [Fig. 8(b)]. The full list of discriminatory wavelengths can be found in Appendix, Tables 7 and 8.

Fig. 8

Diagrams illustrating the most discriminatory wavelengths for all features of interest for each feature binary classification task for unnormalized (a) and AUC normalized (b) spectra. Spectral features of interest are indicated by tickmarks along $x$-axis—a total of 27 marks for each classification task; the $y$-axis shows wavelengths.

To determine spectral regions with the most diagnostic relevance, the entire wavelength range (450 to 750 nm) was divided into 20 nm spectral bands and the frequency of appearance of discriminatory wavelengths in each band was evaluated (Table 3). For both unnormalized and area normalized spectra, wavelengths associated with hemoglobin absorbance between 510 and 610 nm appeared frequently in discrimination between normal and benign versus dysplasia (MD and SD) indicating that blood absorption plays an important role in discrimination of histologically normal and abnormal tissue. The wavelength band of 450 to 469 nm was also prominent in all diagnostic categories for unnormalized features and especially in the separation of benign from MD and SD. This trend can be attributed to a strong superficial scattering of light in the blue spectral range from the surface keratin layer. Indeed, average spectra for the benign category shown in Fig. 6 generally exhibited a steeper, more negative slope in the wavelength range 450 to 500 nm as compared to other diagnostic categories. However, the wavelength band of 450 to 469 nm did not appear frequently for the area-normalized spectra indicating that this wavelength region was strongly associated with signal amplitude rather than spectral shape. Prevalent discriminatory wavelengths for the discrimination of MD from SD were not associated with hemoglobin absorption indicating that this classification was more sensitive to differences in scattering. Indeed, the wavelength region above 600 nm is highly significant in binary tasks associated with keratinized tissues, such as benign, MD, and SD. It could be associated with better penetration of red-NIR light through the keratinized layer in these sites that results in the collection of more diagnostically relevant information.

Table 3

Wavelengths regions with the greatest appearance for each classification task that are listed in order of appearance frequency. Only frequencies above 10% are listed.

After determining the most discriminatory wavelengths, we carried out selection with mRMR that included all features listed in Table 1 for all detection fibers to identify the best combination of spectral features for each diagnostic classification task (Table 4). Features selected by the mRMR algorithm included scattering signals from all detection fibers as well as features associated with polarization gated and diffuse scattering spectra; this implies that scattering signals collected from various depths were found to be diagnostically relevant. The discrimination of normal tissue from dysplasia (SD, MD, and MD and SD combined) and the benign category produced excellent results with AUC values close to 1 (Table 4 and Fig. 9). However, the discrimination of the more clinically challenging classification of the benign from dysplasia was more challenging as reflected by significantly lower AUCs. The poorer performance in discriminating the benign and dysplastic categories could be attributed to keratinization confounding the diagnosis. Indeed, an increase in scattering is expected for MD and SD cases due to alterations in epithelial morphology, such as increased nuclear size, hyperchromasia, and pleomorphism along with keratin formation. However, benign sites are also associated with an increased scattering signal due to a high degree of keratinization (Fig. 4). A strong scattering by the keratin layer can create a significant background scattering, thus, interfering with detection of scattering signatures of dysplasia.

Table 4

Features found to be important in discriminating between diagnostic categories using mRMR and the corresponding AUC. Features identified with a wavelength (nm) refer to intensity at the most discriminatory wavelength, and those labelled with norm are associated with normalized spectra. No wavelengths are listed for spectral features associated with the spectral mean—the average intensity taken across the entire wavelength range.

Fig. 9

ROC curves for each classification task.

After conducting the global analysis with all detection fibers, we addressed the question of whether the multifiber design of this OPRS probe improves diagnostic performance as compared to any given single fiber pair of the probe. Similar to the multifiber analysis, the most diagnostically relevant spectral features and the corresponding AUCs for each parallel/perpendicular pair of detection fibers were determined using feature selection with mRMR (Table 5). With global analysis, including all detection fibers, the separation of normal from MD and SD resulted in excellent AUCs for all fiber pairs with no significant difference between different pairs or their combination. However, discriminations of the benign from MD and from SD categories significantly worsened as compared to the combination of all fibers. These results were further supported by the statistical overtraining analysis described below.

Table 5

Features associated with individual fiber pairs found to be important in discriminating diagnostic groups using mRMR and the corresponding AUCs.

3.5.

Check for Overtraining

LOOCV together with a permutation test wherein the data were randomly assigned a diagnostic class was used to check for overtraining. The results from the permutation test are shown in Fig. 10. The $\text{mean}\pm \text{standard deviation}$ of the AUCs for the data with randomly shuffled diagnoses are shown as black lines and are compared with real AUCs from Tables 4 and 5 obtained for the correctly classified dataset. The real AUCs in classifications of the normal from the benign and dysplastic cases were well above the errors bars of random permutation tests confirming statistical significance of data analyses. However, there was no significant difference between data analyses that included all fibers or any separate individual parallel/perpendicular detection fiber pair. The benign versus MD and SD classification tasks were also statistically significant in the case of the global analysis that included all detection fibers, but it was not the case for any of the individual detection fiber pairs. The real AUCs of the MD versus SD classification task overlap with the error bars of the shuffled AUCs that indicates overtraining of the dataset. Therefore, the classification task failed to discriminate MD and SD sites.

Fig. 10

Permutation test to check for overtraining. Collected spectra were randomly assigned diagnoses while keeping the overall distribution of diagnostic categories the same; samples were randomly shuffled 100 times. The mean and standard deviation of the AUC obtained from the shuffling are presented and compared to the real AUCs shown as stars (all fibers combined), dots (BF1 fiber pair), squares (BF2), and circles (BF3).

We also performed analyses using a dataset that did not include features extracted from area-normalized spectra (Appendix, Fig. 12). It is interesting to note that no statistical significance was achieved in discrimination of the benign from MD and SD categories using only unnormalized data.