2.1.

Study Design

Figure 1 shows a schematic of the study design. As indicated in the figure, there were three spectrometers, labeled FEEM̱3(1), FEEM̱3(2), FEEM̱3(3), referred to as spectrometers MDA, LBJ, and BCCA, respectively. Each of the spectrometers is in a different location: FEEM̱3(1) is located at the Colposcopy Clinic of the University of Texas M. D. Anderson Cancer Center in Houston, Texas (MDA); FEEM̱3(2) is located at the Lyndon B. Johnson (LBJ) County Hospital in Houston, Texas; and FEEM̱3(3) is located at the Colposcopy Clinic of British Columbia Cancer Agency in Vancouver, British Columbia, Canada (BCCA). The spectrometers are difficult to move, so we left them in their respective locations and brought all other materials to them. Each spectrometer has a standards tray, which contains 13 optical standards, which were shipped between sites for this study. The fiber optic probes were generally attached to particular spectrometers, but we shipped them between sites as well. Every fiber optic probe, standards tray, and the calibration standards themselves were labeled prior to conducting the study.

Fig. 1

Study design for the full factorial analysis.

The study was designed by the statisticians and principal investigator (DC, MF). The study involved the three spectrometers, four fiber optic probes, and three sets of optical calibration standards. There were a total of 4 $(3\text{spectrometers}+\text{repeat}\text{of}\mathrm{LBJ})\times 3\left(\text{optical}\text{calibration}\text{trays}\right)\times 4\left(\text{fiber}\text{optic}\text{probes}\right)\times 2\left(\text{days}\right)\times 3$ (measurements within each day) = 288 measurements made on each of the 13 individual optical calibration standards. Measurements were made by one research assistant (OS) in the dark, assisted by a senior clinical biomedical engineer (RP). The data, consisting of approximately $140\mathrm{Gbytes}$ , was quality assured by the codirector of the instrumentation core (BP) and the senior programmer (DS). The first analysis reported by Lee involved the raw data. The subsequent analyses reported here were carried out by the codirector of the instrumentation core, the programmer, and the director of the biostatical core and the research assistant (BP, DS, DC, and OS). The ramifications of the analysis were discussed with the codirector of the instrumentation core (NMcK) and other members of the instrumentation team (CMacA and RRK).

2.2.

Use of Spectroscopic Measurements

Fluorescence spectroscopy involves illuminating the tissue sample with excitation light and measuring the emitted light at longer wavelengths. Our devices use light delivered through a fiber optic probe that also collects the light that is fluoresced. The data are stored as an intensity at each excitation-emission wavelength. The fluorescence in tissue comes from several biologically relevant fluorophores including NADH, FAD, tryptophan, elastin, and collagen. Absorbers, such as hemoglobin, also contribute to tissue fluorescence by absorbing excitation light and remitted fluorescence. There has been some success using fluorescence spectroscopy to detect cancer and precancer.^{8, 9, 10, 11, 12}

An important issue in the successful implementation of any optical device is calibration of the spectroscopic system and control of extraneous factors. Control of these factors in spectroscopy depends on the timely measurements of standards. In the following sections, we describe the optical instrumentation and calibration standards used to control and study these factors.

2.3.

Spectroscopic Instrumentation

The fast excitation emission matrix (FastEEM) systems are spectroscopic devices for measuring the intensity of fluorescence at a variety of excitation and emission wavelengths. Typically, the excitation light is conducted through a fiber optic cable to a probe to illuminate the sample. The fiber optic probe itself is fairly small (diameter $\sim 4\mathrm{mm}$ ). At the distal tip of the fiber optic probe, a mixing element that consists of a large diameter optical fiber uniformly illuminates the sample with the excitation light and collects the emitted light from the same area. The emission light is then conducted through another set of fibers in the probe back to a diffraction grating, which separates the wavelengths and projects the emitted light onto a charge-coupled device (CCD). There are a number of components of such a system which can vary under temperature, age, and other factors, thus requiring highly precise calibration by measuring standards with known optical response.

Figure 2 shows a block diagram of the FastEEM3 spectrometer. Broadband illumination is provided by a xenon arc lamp, and a set of filters selects the desired part of the excitation spectrum. Fluorescence is measured at excitation wavelengths from $300\text{to}530\mathrm{nm}$ in increments of $10\mathrm{nm}$ , with a FWHM of $7\mathrm{nm}$ .

Fig. 2

Schematic of fast-EEM3. Xe LS, xenon light source; ${\mathrm{F}}_{\mathrm{ex}}$ , fluorescence excitation filter wheels; ${\mathrm{F}}_{\mathrm{ex}}$ , TS, translation stage; P2, sampling powermeter; P1, probe powermeter; T, tungsten light source; M, mercury-argon light source; IS, integrating sphere; ${\mathrm{F}}_{\mathrm{em}}$ , fluorescence emission filter wheel; Spx, spectrometer.

The exterior of the fiber optic probes for FastEEM3 are stainless steel, $20\mathrm{cm}$ long, and houses 24 excitation and 12 collection fibers. Probes were tipped with a quartz mixing element $1.8\mathrm{mm}$ in diameter with a length of $20\mathrm{mm}$ which enables roughly uniform illumination of the portion of the sample queried and collection from the same area. We used four probes throughout the course of the study. The four probes were made by three different manufacturers: Multimode (Richmond, Virginia), PNP Optica (Kitchener, Ontario), and Ceramoptec (East Longmeadow, Massachusetts).

2.4.

Standards Measured in the Study

Standards for the calibration of fluorescence include rhodamine (peak emission wavelength at $579.5\mathrm{nm}$ ), exalite (peak emission wavelengths at 370 and $400\mathrm{nm}$ ), and coumarin (peak emission wavelength at $477\mathrm{nm}$ ) as positive standards, and deionized water and a frosted quartz cuvette as negative standards. Three different calibration light sources include a mercury-argon lamp, a deuterium-halogen lamp, and a tungsten lamp to calibrate the wavelength and spectral sensitivity. A National Institute of Standards and Technology (NIST)-traceable tungsten light source is used to determine the spectral sensitivity of the spectrometer. The optical transfer function is calculated and applied to the tissue spectral during the calibration process (step 4 in the following).

In this paper, we analyze the results from the positive standard rhodamine ( $4.2\mu \mathrm{m}$ solution of rhodamine 610 dissolved in ethylene glycol) and the negative standards, frosted quartz cuvette and water. Results from the positive standards, coumarin and exalite, are similar to those of rhodamine and are not presented here. The data were processed using the method described in Ref. 2. The process consists of (1) calibrating the wavelength using the mercury-argon lamp, with an additional wavelength offset correction from the rhodamine peak position, taken from the closest (in time) rhodamine measurement; (2) subtracting the background; (3) dividing the fluorescence intensity by the excitation power meter measurement and exposure time; (4) correcting for the nonuniform spectral response of the fiber optic probe and spectrometer; and (5) normalization by the intensity of rhodamine at an excitation of $420\mathrm{nm}$ and an emission of $579.5\mathrm{nm}$ . Data were smoothed as is done for tissue measurements. Data processing was done with code written in MATLAB (Natick, Massachusetts).

2.5.

Trial Design for Standards Measurements

We took measurements using each of the three FastEEM spectrometers for every possible combination of three calibration standards trays and four fiber optic probes. Measurements were made over $2\text{days}$ for each spectrometer and fiber optic probe combination. The different standards trays were rotated three times in each day because it was unknown whether the variations of the xenon lamp, spectrometer heating, inconsistent contact between the fiber optic probe and standard, or other unknown sources introduced variability within a day or from day to day. Thus, there were a total of 288 measurements made on each of the thirteen individual standards. Additionally, the measurements on the LBJ spectrometer were repeated at the end of the study.

One research assistant made all the measurements. The measurements were obtained in the same order when interchanging the standards trays. Every measurement was timed and recorded into a log file along with detailed descriptions of any problems which were encountered. The problems were discussed daily with members of the instrumentation core.

2.6.

Statistical Methods

The statistical analysis was based on ANOVA, including multivariate ANOVA (MANOVA) and enabled us to quantify the contribution of each factor (and higher order interactions of two or more factors) in terms of the proportion of variance explained and the statistical significance of the factor as a source of variation. All statistical analyses were performed in MATLAB or R (available for free from Ref. 13). The first objective was to perform a MANOVA including the main effects from all factors. However, since the data are hyper-dimensional (each observation has over 4000 intensity values at the different excitation-emission wavelength combinations), one cannot perform a MANOVA on the raw data as the estimated covariance matrix will be singular.^{14, 15} To eliminate this problem, we first performed a principal components analysis (PCA) of the processed data for each optical standard taken during the study and selected enough principal components to capture 99% of the variability so that the principal component data is an accurate approximation of the original data. This reduced the data sufficiently that MANOVA could be performed. Any factors that were found significant were kept for a second stage of the analysis wherein we performed ANOVA on each of the principal component scores to find the percentage of variability explained by each of the significant factors. Then, we computed an overall percentage of variance explained for each significant factor as a weighted average of the percentage of variance explained across the principal components using as weights the percentage of variance corresponding to the principal component.

3.1.

Initial Data Analysis

The variations among spectrometers and optical probes were plotted for both raw and processed data. Figures 3, 4, 5 show the mean raw and processed EEMs of rhodamine, frosted cuvette, and water organized by spectrometer and fiber optic probe. The overall mean is in the right lower plot. Figure 3 shows raw and processed EEMs of rhodamine; excitation wavelengths along the ordinate, emission wavelength along the abscissa, and the ${\log }_{10}$ of the fluorescent intensities are represented with a MATLAB “jet” color map.

Fig. 3

(a) Raw and (b) processed data of the standard rhodamine, a positive standard. EEM plot matrices of the probe study plotted on a log scale. The vertical axis is the excitation wavelength ranging from $300\text{to}530\mathrm{nm}$ (tick marks at $46\mathrm{nm}$ intervals). The horizontal axis represents the emission wavelengths ranging from $300\text{to}800\mathrm{nm}$ (tick marks at $100\text{-}\mathrm{nm}$ intervals). Each individual box-probe combination EEM is the mean of six measurements: three measurements per day for $2\text{days}$ . The first four columns represent the results from the probes, while the first four rows represent the four box measurements. The right lower quadrant is the overall mean of both types of equipment.

Fig. 4

(a) Raw and (b) processed data of the standard water, a negative standard.

Fig. 5

(a) Raw and (b) processed data of the standard frosted cuvette, a negative standard. EEM plot matrices are in the same arrangement as in Fig. 3.

The raw data, shown in Figs. 3a, 4a, 5a, demonstrate significant differences in the data. The most prominent differences of the raw data are (1) light source intensity of the spectrometers, (2) the transmission efficiency of the fiber optic probes, and (3) the “noisier” BCCA spectrometer.

A source of the raw intensity differences when comparing the boxes is due to the illumination intensity of the xenon light source. All xenon bulbs were of the same model, but the intensity decreases with usage. Measurements demonstrated that the xenon light source intensity at BCCA and LBJ were at approximately 10 and 30% of the MDA box, respectively.

The reduced raw intensity of the MDA1 and, to a lesser extent, the LBJ fiber optic probes are due to their reduced optical transmission despite the same specifications given to the various probe manufacturers. The transmission of the probes as compared to the BCCA probe are 26, 89, and 47% for the MDA1, MDA2, and LBJ probes, respectively, at $400\mathrm{nm}$ .

To compare the CCD noise of the three FastEEM3 spectrometers, data were recorded from each with all shutters closed to monitor the CCD dark current and readout noise. Although all three CCDs were within their noise specifications, marked differences in performance were evident. The mean counts per pixel of the BCCA, LBJ, and MDA CCDs are 445.23, 511.73, and 475.68, respectively. The coefficients of variation, which describe the variable noise, of the BCCA, LBJ, and MDA CCDs are 0.081, 0.095, and 0.057%, respectively.

3.2.

Processed Data Analysis

Processing the data, as shown in Fig. 3b, 4b, 5b, removes the majority of the differences in the xenon lamp and spectrally uniform probe attenuation. The variation of the data caused by lamp intensity and probe transmission after processing is dramatically reduced, but the marked differences of autofluorescence in the fiber optic probes then becomes the major source of variation. The processing step 4 (Sec. 2.4), which corrects for the nonuniform spectral response of the fiber optic probe and spectrometer is unable to account for the fiber optic probe autofluorescence. The MDA1 fiber optic probe has a marked autofluorescence compared to the other probes used in this study and is evident in the processed plots. Our group is investigating whether the fiber optic probe autofluorescence can be removed from the previously measured tissue spectroscopic measurements. Additionally, only the MDA1 fiber optic probe was used on an appreciable number of patients. The vast majority of patients $(\sim 80\%)$ were measured using the BCCA probe. Due to the variability among probes evidenced in this study, we isolated the problem and are now constructing the fiber optic probes in our lab for further clinical trials. We implemented a stringent quality control process in the manufacturing of the fiber optic probes.

3.3.

ANOVA Analysis

The ANOVA analysis was conducted on a dimensionally reduced data set obtained¹⁴ by PCA because the actual data have many more variables than observations. In every case, we selected enough principal components (PCs) to capture 99% of the variability. Table 1 shows the number of PCs required to explain 99% variance. For rhodamine, we see that one PC in the raw data captures at least 99% of the data. This component is essentially the shape of the rhodamine spectrum and the corresponding PC score is the amplitude. There are large amplitude variations between devices, but these are mostly removed by the processing and one sees that it requires 33 PCs to explain 99% of the variance in the fully processed rhodamine. The large number of components required indicates that the overall variability is less.

Table 1

The number of PCs required to explain 99% variance.

Once the dimensionalities of the data were sufficiently reduced we conducted a standard multivariate ANOVA using Wilk’s test statistic.¹⁴ Each factor was included only as a main effect in this analysis. The results are presented in Table 2 in which we examine the $P$ values. We see that with one exception, only the spectrometer and fiber optic probe are significant sources of variation. The one exception is that for fully processed rhodamine, the standards tray is a significant source of variation. In this case, the percentage variance explained is less than 1%. Thus, in general, most of the standards trays, day-to-day variation, and order of measurement are not statistically significant. For all factors, we further analyzed the percentage of variance explained, including all possible interactions, as shown in Table 3 . These results show that the processing reduces most of the variability due to these extraneous factors. For the frosted cuvette, the main effect due to the spectrometer accounts for 72% of the variance in raw data, but goes down to 1% in the fully processed data. Thus, the processing eliminates most of the effect due to the spectrometer alone. Similarly for the effect due to the spectrometer, the percent variance explained in water goes down 82% before processing to 0.5% after processing, and in rhodamine from 56 to 9%. The effect due to probe alone shows similar but less dramatic differences (11 to 6% for the frosted cuvette and 21 to 4% in rhodamine). The one case in which processing increases the variance explained is water where the fiber optic probe effect rises from 5 to 11%. The increased variance of the probe contribution is explained by the autofluorescence of the fiber optic probes. Note the dramatic decrease in percentage of variance explained for the spectrometer after processing, which is consistent with reducing the differences in lamp output of the spectrometers. With processing, this variation nearly disappears and the percent variance explained by the autofluorescent probe takes precedence with this negative standard.

Table 2

The P values from the MANOVA for optical standards on frosted cuvette, water, and rhodamine.

This statistical analysis also included interaction terms. For example, the spectrometer-fiber optic probe interaction term measures how much the mean for a particular spectrometer and fiber optic probe combination differs from the predicted value obtained by adding the mean for that spectrometer and the mean for that fiber optic probe. For all three standards, there was substantial reduction in this term following processing. In the case of fully processed rhodamine where the standards tray was significant, we have included all the interactions of this factor with spectrometer and fiber optic probe as well as the three-way interaction, which describes the output that can not be predicted from either the three individual components or one component and a two-way interaction.

It is interesting to look at the rows of each type of measurement in Table 3. These rows sum to less than 100%, with the remainder being the unexplained variation (or “mean square due to error”). For instance, in raw rhodamine the row sum is 99%, indicating that almost all of the variability is due to the spectrometer, fiber optic probe, and their interaction. For fully processed rhodamine, the spectrometer alone accounts for 9% of the variance explained, the fiber optic probe for 4%, the standards tray for less than 1%, the spectrometer-fiber optic probe for 5%, the spectrometer-tray for 2%, the fiber optic probe-tray for 1%, and the spectrometer-fiber optic probe-tray for 5%.

3.4.

Contrast Plots

Figure 6 presents contrast plots of the processed rhodamine. Contrast plots are used to emphasize the differences from the mean of the data set. The contrast plots of the spectrometer (or fiber optic probe) means are computed by subtracting the overall mean from the processed data of the spectrometer (or fiber optic probe) mean. The individual spectrometer-fiber optic probe combination contrast plots are computed by subtracting the corresponding spectrometer mean and fiber optic probe mean from the spectrometer-fiber optic probe combination mean and adding the overall mean. The contrast plots highlight the differences in the processed rhodamine at excitation wavelengths 300, 510, 520, and $530\mathrm{nm}$ . The marked discrepancies at these excitation wavelengths led us to perform the statistical analysis using only excitation wavelengths $310\text{to}500\mathrm{nm}$ and is termed “reduced EEM” in the tables. The reduced EEM of rhodamine decreases the variance explained from 9 to 4% for the spectrometer and is decreased to the level of the effect of the fiber optic probe. This analysis suggests that the data generated from excitation wavelengths 300, 510, 520, and $530\mathrm{nm}$ with our current instrumentation are variable between spectrometers. At an excitation wavelength of $300\mathrm{nm}$ we expect this to be due to the low intensity of rhodamine, but we are unsure of the cause at the higher excitation wavelengths. However, our algorithm does not use the aforementioned excluded excitation wavelengths in our current study of cervical neoplasia.

Fig. 6

Contrast plots of processed rhodamine in the same arrangement as Fig. 3. The white regions are where the comparisons ares nearly equal, while the gray area contain no EEM data. Note the marked differences in the extreme lower $\left(300\mathrm{nm}\right)$ and upper $\left(510\text{to}530\mathrm{nm}\right)$ excitation wavelengths.

Table 3

Percentage of variance explained by each factor and their interactions for optical standards frosted cuvette, water, and rhodamine.