2.1.

Calibration

The fundamental role of standards is to provide a sound physical basis to map the measured signal to the true response of the sample being measured. All changes of energy originating from sources other than the measured sample are considered system-dependent signals that must be detected and corrected using the appropriate calibration standards. Standards also quantify unwanted background signal levels and facilitate background detection and removal. It is important to use both positive standards (standards with a known, strong optical signature) and negative standards (standards where no response is expected) to enable this calibration. Standards that bracket all data dimensions, including time, are required to ensure accurate translation from measured data to calibrated results. Careful consideration of the nature of standards and frequency of their measurement is necessary to guarantee that all dimensions are calibrated correctly.

2.2.

Quality Assurance

Standards are also used to track the device response as a function of time to mitigate various operational issues that may change over time. Standards provide a well-known blue print of expected (ideal) results. Measurement and analysis of data from appropriate standards with sufficient frequency can be used to easily identify any variation from this blue print; this can be used to correct previously acquired data and to modify instrumentation to prevent similar problems in future data acquisition.

The calibrated FastEEM data should become device independent and should be able to be combined for analysis with data acquired with other calibrated optical devices. ^{8, 16, 17, 18, 19} In this study, two generations of the FastEEM system were tested clinically at three different sites: the University of Texas (UT) M.D. Anderson Cancer Center in Houston (FastEEM2); the Lyndon B. Johnson Harris County Hospital District in Houston, Texas (FastEEM2); and the British Columbia Cancer Agency (BCCA) in Vancouver, Canada (FastEEM3). The Institutional Review Board of the UT at Austin, M.D. Anderson Cancer Center in Houston, and the BCCA in Vancouver approved the conduct of this study, and all subjects gave written, informed consent before study participation.

2.3.

Measured Standards

It is critical to have a comprehensive set of standards in order to accommodate all data dimensions and sources of measurement variability like stages, filters, and shutters controlling integration times. With each device and at each clinical site, data were acquired from a series of standards as well as from patient sites. Tables 1, 2 detail all positive and negative standards measured with each device.

Table 1

FastEEM2 calibration standards.

These standards should be measured frequently in actual clinical settings prior to clinical trials to anticipate environmental issues, as well as throughout the entire clinical trial. Measurements in the clinic are often performed in small rooms with more than the usual number of personnel. This can sometimes result in temperature, humidity, and room light levels that differ substantially from the laboratory.

2.4.

Positive Standards

The role of positive standards is to provide an input of known value; data measured from the positive standard are compared to measured output. The deviation of the measurement from the known provides correction factors for calibration to remove system-dependent responses. For the FastEEM, the correction factors calibrate for wavelength response, for variations in illumination energy as a function of time and wavelength, and for variations in emission wavelength sensitivity (optical transfer/transduction).

In selecting standards, we looked for those that were easy to obtain commercially and that covered the full UV VIS spectral range in both excitation and emission. Rhodamine, Exalite, and coumarin cover the full range of emission and excitation wavelength of interest and are easy to obtain. The choice of concentration was based on making them optically dilute. We also looked for solvents that were easy to work with and for standards with low photobleaching.

Positive standards measured with the FastEEM2 include: a $\mathrm{HgAr}$ calibration lamp as well as Hg lines from fluorescent room lights (later abandoned) for wavelength calibration purposes, a $4.2\mathrm{\mu }\mathrm{M}$ solution of the organic fluorescent dye, rhodamine 610 (Exciton, Dayton, Ohio) in ethylene glycol to compensate for variations in overall collection throughput, manual measurements of illumination power using a power meter (Newport, Irvine, California, 818-UV) to compensate for variations in the intensity of the excitation light, and measurements of NIST traceable calibrated tungsten and deuterium lamps (550C and 45D, Optronic Laboratories Inc., Orlando, Florida) to correct for nonuniform spectral response of the detection system.

With the next-generation FastEEM3 system, we measured an expanded range of positive standards. These standards include: a $\mathrm{HgAr}$ calibration lamp (Ocean Optics) for wavelength calibration purposes, a $4.2\mathrm{\mu }\mathrm{M}$ solution of the organic fluorescent dye rhodamine 610 (Exciton, Dayton, Ohio) in ethylene glycol, a $2.5\mathrm{\mu }\mathrm{M}$ solution of the organic fluorescent dye coumarin 480 (Exciton, Dayton, Ohio) in ethylene glycol, and a $0.133\mathrm{\mu }\mathrm{M}$ solution of the organic fluorescent dye Exalite 400E (Exciton, Dayton, Ohio) in ethylene glycol to compensate for variations in overall collection throughput. In addition, automated measurements of illumination power were performed using a power meter to compensate for variations in the intensity of the excitation light, and measurements of NIST traceable calibrated tungsten lamp (LS-1-CAL, Ocean Optics, Dunedin, Florida) were performed to correct for nonuniform spectral response of the detection system.

FastEEM3 power meter measurements are acquired at 10-ms intervals, beginning and ending approximately $30\mathrm{ms}$ before and after the shutter open and close time, providing an accurate measurement of actual illumination times and energy delivered. Two power meter measurements are acquired, and used to calculate the energy delivered to the tissue during data acquisition. The first measurement is made from a sampling optical fiber; the proximal tip of the sampling fiber is placed at the same location as the illumination fibers of the fiber optic probe and the distal tip is located at the power meter during tissue measurements. The second measurement is made from the distal tip of the fiber optic probe just prior to patient measurements. Figure 2 shows the location of the sampling fiber (labeled as P1) and the probe output (labeled as P2) in the FastEEM3 system diagram. The measurement from the sampling fiber is used to determine the optical power delivered during each tissue measurement, and the measurement from the probe output is used to calibrate the ratio of the sampling fiber measurement to the probe output.

2.5.

Negative Standards

The goal of measuring negative standards is the detection of light introduced into the illumination and measurement optical paths due to leakage, stray light, or component fluorescence or contamination that can result in measurement artifacts. FastEEM2 and FastEEM3 negative standards measured include: a measurement made with the illumination source shuttered and the probe placed on the sample (background), a measurement made with the camera shutter and the illumination shutter closed (dark current), and finally two measurements made under the same measurement acquisition parameters (integration time, illumination wavelength, etc.) conditions as tissue, but first with the probe placed in a bottle of distilled water and second with the probe placed in contact with the frosted quartz cuvette. The purpose of these last two measurements is to use a sample with no intrinsic fluorescence to provide a check for device related autofluorescence. The water sample results in only small specular reflection due to the refractive index difference at the probe tip and water interface with the remaining light going forward with minimal backscattering. The frosted cuvette is expected to diffusely backscatter the illumination light in a similar way to epithelial tissue. This enables us to monitor device related autofluorescence, which could be backscattered by tissue and contaminate the measurement. Our criteria were to reject data if the tissue signal drops below 10 times the background fluorescence.²⁸ Device-related autofluorescence can sometimes be comparable to tissue signals if proper materials are not used in fiber optic probe construction or as a result of defects or damage accumulated over time.

2.6.

Frequency of Standards Measurements

It is important to measure standards frequently enough to capture any drift in the performance of the measurement system over time. A typical clinic day with the FastEEM2 system would result in measurements from one to two patients. Prior to each patient measurement, a full set of positive and negative standards would be run. A typical day in clinic with the FastEEM3 would result in measurements from two to three patients. A full set of standards would follow the initial 20-min device warm up. Throughout the course of the day Rhodamine measurements would be repeated every $2\mathrm{h}$ . As we learned about system performance issues during the course of these trials, we adjusted the frequency with which standards were measured.

2.7.

Analysis of Measured Standards

Measured standards were analyzed in three ways—first, to process data from positive standards and tissue to calculate fluorescence spectra in calibrated units; second, to verify the calibration process by comparing processed data from positive standards to expected results; and third, to identify and quantify any measurement artifacts, providing a means for data recovery under adverse events.

In the next section, we detail each step in the processing algorithm based on these standard measurements and the quality assessment tools developed to assess each type of standard measurement. In each case, statistical analysis over the timeline of each collected standard was used to track the performance of the FastEEM system. Descriptive statistics including number of samples, arithmetic mean and standard deviation, median, minimum, and maximum were tabulated for the raw data, calibrated data, and calibration parameters. In addition, raw spectra were plotted over time, 10 measurements per figure to study consistency of spectral shape of measurements. These data enabled expert reviewers to identify changes in the instrument performance; these changes were then correlated with specific physical phenomena based on experts’ background knowledge and events related to device maintenance or calibration recorded in the FastEEM operator’s logbook.

2.8.

Raw Data Processing Algorithm

Raw data recorded with the FastEEM are processed to yield system-independent data in a six-step process. Figure 4 illustrates the generalized processing algorithm flow chart applied to data acquired with both FastEEM systems.

Fig. 4

FastEEM data processing flow chart.

2.9.

Additional Device-Dependent Corrections (Standards-based Correction)

When combining data acquired from multiple devices, it is important to use identical components and calibration standards. Given the length of the clinical trial, it may be impossible to replace a failed component with an identical counterpart due to discontinued production or manufacturers ceasing production. Two additional corrections were necessary to compensate for device differences. The first correction was due to the differences in the components used for power measurements. Using the FastEEM3 light source, power measurements were made at each excitation with the FastEEM3 powermeter and the FastEEM2 powermeter. The powermeter correction factor calculated by dividing the FastEEM3 by the FastEEM2 power measurements was applied to the FastEEM2 data only. The second correction applied was due to system response differences that could not be captured by the tungsten-based correction factors already applied. Only nine tungsten measurements were made with the FastEEM2 device at the beginning of the trial. Before new measurements could be made at the end of the trial, the FastEEM2 detector failed. An additional correction was devised using the positive standards, which were measured later in the trial. Spectra at each excitation for Exalite, coumarin, and rhodamine were concatenated to cover the entire emission range in use. A standards-based system response correction factor was then computed by dividing the FastEEM3 standard spectra by the FastEEM2 standard spectra; the resulting correction factors were applied to the FastEEM2 tissue data.

2.J.

Wavelength Calibration Validation

The positive standard of rhodamine was used to validate the accuracy of the wavelength calibration process. The emission maxima of this fluorophore occur at $580\mathrm{nm}$ independent of excitation wavelength. The peak emission wavelength of the processed rhodamine spectra was extracted at two excitation wavelengths (330 and $420\mathrm{nm}$ ). Statistical analysis of peak locations between measurements at each excitation wavelength provided a tool to assess the consistency and accuracy of the wavelength calibration algorithm.

2.K.

Wavelength-Dependent Throughput Calibration Validation

The accuracy of the wavelength-dependent throughput calibration was assessed by comparing the shape of the processed emission spectrum of the positive standard rhodamine to that measured with two other commercially available calibrated spectrometers (SPEX, Fluorolog II Spectrometer, Edison, New Jersey, and Photon Technologies International Quanta Master Model C Spectrofluorometer, Lawrenceville, New Jersey). To assess unexpected drifts in the wavelength-dependent system throughput, tungsten spectra were plotted over time.

2.L.

Illumination Energy Calibration Validation

The processed emission spectra of the positive standard rhodamine were assembled into an excitation emission matrix and the excitation spectrum was extracted at 580-nm emission. The shape of the excitation spectrum was compared to that measured with two other commercially, available calibrated spectrometers (SPEX, Fluorolog II Spectrometer, Edison, New Jersey, and Photon Technologies International Quanta Master Model C Spectrofluorometer, Lawrenceville, New Jersey). The shape of the excitation spectrum was assessed over time throughout the trial to identify potential problems with the illumination energy calibration process.

2.M.

Engineering Analysis Methods

The statistically derived calibration coefficients and standard spectra collected throughout the trial were analyzed to identify any instrumentation or processing problems, and to formulate a course of action to prevent future occurrences and correct affected measurements, which had already been collected. There is an ethical responsibility to the patients, the funding agencies, and all health care providers involved to use every patient measurement made. Most instrumentation problems can be corrected at the hardware and/or software level. It is important to correct hardware failures to prevent reoccurrence and/or acquisition of potentially unusable data. It is critical to devise algorithms based on physical models of cause and effect that can recover otherwise unusable data.

The raw and processed standards data collected throughout the trial were analyzed in two simple steps. First measurements are compared to ideal values. The spectral response of each positive standard is known. By comparing measured to ideal, automatic detection of deviation from the ideal can be implemented. This also allows for calculation of correction factors for affected measurements, which have already been collected. In the previous step, the symptom of the failure is identified. Correlating this symptom to a physical cause requires domain knowledge, well-documented device event logs, and sometimes experimentation. The extent of the affected measurements can be determined based on the frequency of occurrence of the symptom and validated based on the nature of the cause.

3.1.

Measured Standards and Their Frequency

Data were acquired from 1295 sites in 442 patients using the FastEEM2 device and from 788 sites in 322 patients using the FastEEM3 device. The frequency with which standards were measured was increased with the FastEEM3 as a direct result of experiences in the FastEEM2 study.

3.2.

Raw Data Processing Algorithm

The data processing algorithm was implemented in Matlab (Mathworks, Inc., Natick, Massachusetts) Ver. 6.5 Rel. 13 on the Windows XP operating system. All data were stored and processed on portable fire-wire hard drives. Processing algorithm performance was limited by the file (i/o) access time. Raw data acquired from standards and tissue were processed to yield data in the form of excitation emission matrices (EEMs), encapsulated postscript files of processed EEM images in line plot and contour format, a comprehensive QC file with an image of every processed EEM in line plot and contour format, and processing logs including errors and lists indicating which standards data were used to process each tissue measurement. EEMs contain calibrated fluorescence intensity as a function excitation and emission wavelength.

Figure 5 shows the result of processing data acquired with both systems from the positive standard rhodamine at $420\mathrm{nm}$ excitation at each step in the processing algorithm. Figure 5a illustrates the effect of dark current and background subtraction between FastEEM2 and FastEEM3. Figure 5b shows the emission spectra after wavelength calibration, energy normalization, and smoothing with Savitzky-Golay filtering. Figure 5c shows the final processed emission spectra. Following processing, emission spectra have the same peak emission wavelength and spectral line shape; the small differences observed with this positive standard provide a measure of the calibration accuracy. We next describe how data processed in this manner were validated.

Fig. 5

Normalized rhodamine emission spectra from FastEEM2 and FastEEM3 at 420-nm excitation following various stages of data processing: (a) emission spectra after background subtraction; (b) emission spectra after wavelength calibration, energy normalization, and filtering; (c) final processed emission spectra (c.u. = calibrated units).

3.3.

Wavelength Calibration Standard Validation

Figure 6 shows the consistency of the wavelength calibration derived from the $\mathrm{HgAr}$ measurements using the FastEEM3 device over the lifetime of the trial. Figure 6a shows ten consecutive $\mathrm{HgAr}$ spectra measured in March to April 2003. The first five mercury peaks in the measured spectrum (365, 404.7, 435.8, 546.1, and $578\mathrm{nm}$ ) are used to derive a linear calibration between pixel and wavelength. Figure 6b shows the position of the pixel corresponding to each peak over the duration of the clinical trial. The pixel positions are relatively constant over time, except for three discrete steps, which correspond to changes in grating incidence angle made at the beginning of the study to improve efficiency and to a change in magnification due to camera realignment. Figures 6c and 6d shows the slope and intercept of the linear calibration coefficients as a function of time throughout the study. The changes in grating angle and magnification changes can be clearly seen.

Fig. 6

Graphical tools developed to validate wavelength calibration of the FastEEM systems: (a) 10 consecutive, overlaid spectra of the $\mathrm{HgAr}$ lamp acquired with FastEEM3; (b) pixel locations of the five mercury peaks used for wavelength calibration as a function of time throughout the clinical trial; and (c) slope and (d) intercept of the linear relationship between pixel number and wavelength as a function of time throughout the clinical trial.

3.4.

Wavelength Calibration Algorithm Validation

Statistical analysis of the emission spectra of rhodamine showed that the peak emission wavelength did not occur at the same value for all excitation wavelengths within an EEM measurement as expected. Furthermore, the value of the peak emission wavelength shifted from one measurement to another [Fig. 7a ]. Engineering analysis showed that these shifts were associated with thermal changes in the position of the spectrometer grating, which affected the wavelength calibration. These thermal changes shifted the spectrum slightly across the face of the CCD but did not affect the dispersion of the spectrometer; thus, only the intercept of the wavelength calibration and not the slope was affected. A particularly troubling thermal effect was a thermal expansion and contraction that was sourced to the optomechanical mounting of the grating in the Jobin Yvon Triax 320 spectrographs used in the FastEEM3 system. As the system warmed up by about $10°\mathrm{C}$ during operation, movement of the grating shifted spectral peaks by up to 11 pixels in the wavelength direction [Fig. 7b]. Additionally, a vertical shift of up to 6 pixels was also observed. This resulted in wavelength calibration errors of up to $4\mathrm{nm}$ . Because a $\mathrm{HgAr}$ calibration spectrum was obtained only once per day, this standard could not be used to correct for this thermal drift. Rhodamine spectra were collected approximately every $2\mathrm{h}$ . Monitoring the pixel location of the rhodamine peak provided a measure to correct for this thermal drift. Figure 7c shows the rhodamine spectra processed using this correction. Note that the wavelength position of the rhodamine peak is now constant.

Fig. 7

(a) Ten consecutive emission spectra of rhodamine at 330- and 420-nm excitation processed with the initial wavelength calibration algorithm. Note that the peak emission wavelength shifts by approximately $\pm 3\mathrm{nm}$ . (b) Pixel location of 546-nm mercury peak as a function of time following initial cold start of FastEEM3. The Xe arc lamp was turned off at $260\min$ . This effect was traced to thermal expansion and contraction of the grating mount. (c) Ten consecutive emission spectra of rhodamine at 330- and 420-nm excitation processed with the final wavelength calibration algorithm that corrects for temporal variations associated with thermal drift.

3.5.

Intensity Calibration Validation

Tungsten standards were used to calculate system response correction factors. Consecutive tungsten spectra were plotted over time to verify the consistency of the wavelength-dependent system response throughout the trial. Figure 8a shows resulting rhodamine spectra at $420\mathrm{nm}$ excitation acquired with FastEEM2 and FastEEM3 after processing to correct for the system response, as well as spectra measured with the calibrated laboratory spectrofluorimeter (Spex Fluorolog II). The spectral shape agrees well in all cases. Figures 8b and 8c show similar data for two other organic fluorescent dyes, Exalite at 320-nm excitation [Fig. 8b] and coumarin at 400-nm excitation [Fig. 8c]. This confirms the calibration across a broad wavelength range.

Fig. 8

Comparison of final processed emission spectra obtained with the FastEEM2, FastEEM3, and reference spectrometer for organic fluorescent dyes: (a) rhodamine at 420-nm excitation, (b) Exalite at 320-nm excitation, and (c) coumarin at 400-nm excitation.

3.6.

Illumination Energy Calibration

To assess whether the illumination energy calibration was effective, we compared the average excitation spectra of the Rhodamine standard at 580-nm emission measured with both devices [Fig. 9a ]. Figure 9b shows the ratio of the average Rhodamine excitation spectra for both systems. The ratio between FastEEM3 and FastEEM2 is close to 1 over the entire excitation wavelength range.

Fig. 9

(a) Average rhodamine excitation spectra at 580-nm emission measured with FastEEM2 and FastEEM3, where error bars represent one standard deviation, and (b) ratio of average rhodamine excitation spectra at 580-nm emission measured with FastEEM3 to that measured with FastEEM2.

3.7.

Final Validation

The true test of measurement device independence is whether patient data collected from a large and representative group of patients with different devices can be combined without loss of information. In this study, we measured fluorescence EEMs from 764 patients. We compared the average spectra of squamous normal tissue acquired with the two FastEEM systems. Figure 10 shows the average squamous normal tissue spectra for at 330-nm excitation [Fig. 10a] and 360-nm excitation [Fig. 10b] for both systems. The difference in starting wavelength is due to differences in excitation filter characteristics for both devices. The shape and intensity of both spectra are comparable at each excitation. The spectra shown in [Fig. 10a] are an average of acquired fluorescence at 330-nm excitation for each device without normalization. Error bars are very large (not shown) due to biological differences between patients like menopausal state that can amount to more than an order of magnitude.³⁰ Given the 10-fold patient-to-patient variation in fluorescence intensity, we feel that these results demonstrate excellent agreement.

Fig. 10

Average emission spectra of squamous normal cervical tissue measured in vivo from 435 sites in 206 patients with FastEEM2 and 167 sites in 98 patients with FastEEM3 at (a) 330- and (b) 360-nm excitation.