2.1.

Confocal Scanner

As with fluorescence microscopes, the excitation beam enters the system by reflecting from a dichroic splitter. The excitation is provided by a pulsed 488- or 405-nm laser. As explained below, the pulse length is matched to the propagation time through the delay lines shown in Fig. 1. Averaged over 1 s, excitation levels at the sample are typically in the 1 to 2 mW range. Optical power should be limited to about 5 mW to avoid potential tissue damage.⁸ As a result, compact diode lasers are appropriate—in our case a PhoxX laser from Omicron, Rodgau-Dudenhofen, Germany.

Confocal microscopes are widely available from several leading companies. The basic concept of confocal microscopy is shown by the two scanning mirrors and associated optics at the left side of Fig. 1. As the two mirrors are scanned in an $x–y$ raster pattern, the focused spot on the sample is always conjugate with a pinhole or small-area detector (the pinhole in Fig. 1 is the fiber aperture marked with an asterisk). The pinhole limits the acceptance of light emitted from axial and lateral locations away from the position of the focused spot. Since fluorescence from biological materials typically occurs within 1 to 5 ns after excitation, both reflected and fluorescence signals propagate back to the pinhole before the mirrors can move appreciably. This allows the mirrors to de-scan back-propagating photons such that light from the focused spot indeed stays conjugate with the pinhole. As a result, scanning confocal systems can generate maps (or “images”) with spatial resolution at or near the laser spot size. For the microscopy mode depicted in Fig. 1, the 488-nm laser beam is focused at $\mathrm{NA}=0.24$ to a 1.2-μm diameter spot and scanned at $10\text{frames}/\mathrm{s}$ over an adjustable field (that can be zoomed from 50 to 500 μm). Since the scanning process actually generates a spatial convolution of the focused spot with features in the sample, it is possible to detect features below the diffraction limit (though they appear equal in size to the spot). These ideas⁹ have been applied to images of functional nuclear architecture¹⁰ and have been used for years in the materials and semiconductor industries.¹¹

2.2.

Endoscopic Optics

The system depicted in Fig. 1 performs endoscopy when a sample is placed at the distal end of an endoscope containing 30,000 3-μm diameter fibers. The proximal end of the coherent fiber bundle is placed under the lens. The confocal scanner raster-scans the proximal end of the endoscope. A given fiber within the bundle transfers the laser to the distal end, where a lensed tip (two Grin lenses) relays the beam from the fiber to the tissue. Fluorescence or reflected light from a resolution element is imaged back into the same fiber, propagates back to the proximal end, is collected and de-scanned by the confocal optical system, and is finally focused on the confocal pinhole. This entire process occurs within the dwell time of the raster scan (2.5 μs per pixel in our design). For the endoscopy mode depicted in Fig. 1, the 488-nm excitation beam is focused by the distal tip at $\mathrm{NA}=0.8$ to a 0.4-μm diameter spot and scanned at 10 frames per second over an adjustable field at a working distance of 80 μm (can be zoomed from 10.8 to 108 μm). The Grin lens system relays light from one of the 30,000 fibers into the 0.4-μm diameter spot. Field size is adjusted by changing the angular excursions of the two scanning mirrors. The transmission efficiency of this endoscope is consistent with reports in the literature.¹² The following subsection describes our spectrometer, which conveniently maps proximal reflection and emission bands into different temporal bins.

2.3.

Fiber-Optic Spectrometer

Our fiber optic spectrometer is based on a serial array of reflecting spectral elements (labeled ${\lambda }_1$ ${\lambda }_2$ ${\lambda }_N$ in Fig. 1), delay lines between these elements (fiber loops in Fig. 1), and a single photomultiplier tube. After excitation by a laser pulse, broadband fluorescence from a sample propagates into the array via the confocal aperture. Light of the shortest wavelength band (${\lambda }_1$) reflects from the first element, and light of the $N$’th wavelength band reflects from the $N$’th element. Each wavelength is mapped into a specific time slot. The two-way propagation time in each delay line is equal to the length of the laser pulse (about 166 ns for a 15 bin system). A plot of the detector’s signal versus time contains the laser in the first bin, and then fluorescence of various distinct colors in the subsequent bins. The spectral elements can be fabricated in two ways—fiber tips can be coated with interference filters to create $>10\text{-}\mathrm{nm}$ wide spectral slices, whereas fiber Bragg gratings can be written into the fiber core to create 1- to 2-nm-wide spectral slices.¹³ Most spectrometers employ one grating that disperses light spatially across $N$ detectors or pixels. Our approach employs $N$ tips or gratings that distribute the light temporally against one detector.

When detecting intrinsic fluorescence, the detector in this design sees optical signals in the picowatts to nanowatts range. As a result, the detector must have a high gain–bandwidth product. Detectors with internal gain [photo multiplier tubes (PMTs) at ${10}^5$, electron multiplying charged coupled devices at ${10}^3$, and avalanche photo diodes at 50] are of interest because they can amplify low-level signals above read-out and/or preamp noise. To obtain a useful signal-to-noise ratio (SNR), our system uses a fiber-coupled PMT with ${10}^9\mathrm{V}/\mathrm{W}$ at 6 MHz. The SNR is defined as peak signal divided by the RMS noise. Since our system is shot limited, the SNR is driven by signal strength, which in turn is driven by fluorescence yield, collection efficiency, and spectrometer throughput. In most commercially available confocal instruments, only one or a small number of wavelengths can be measured during each spatial scan. Commercially available systems do offer solutions using a bulk diffraction grating and 32 PMT elements. Our approach is intended to be more cost effective for clinical applications.

Higher collection efficiency is obtained by using a larger confocal aperture. Fiber tip arrays have been fabricated with both 10- and 62.5-μm core fiber. Each tip is attached to another fiber with physical contact such that the coatings are immersed between glass on both sides. The larger core provides $7\times$ higher signals with tissue samples. Neither core size is compatible with fiber gratings for visible wavelengths—which perform optimally with single mode fiber having a 3-μm core diameter. The angular distribution in multimode fiber (MMF) must be assessed regarding operation with coated fiber tips. The angular distribution in MMF is a function of launch conditions.¹⁴ Our confocal optics use low-angle injection—i.e., about $\pm 6\mathrm{deg}$ at the confocal aperture. Due to Snell’s law, the angle of incidence at a fiber tip within the MMF is therefore about 4 deg. In comparison, we should mention that the maximal coupling angle is 24 deg and the maximal internal angle is 15 deg, when a Lambertian source is used to populate all modes in a MMF ($\mathrm{NA}=0.4$). Figure 2 shows 10 modeled high pass filters spaced 30 nm apart. The filter with the shortest wavelength is mapped into the first bin and the filter with the longest wavelength is mapped into the last bin. Modeled spectra show that these curves only move about 2 nm to the blue as the internal angle of incidence increases from zero to 5 deg. The ripple at the bottom of Fig. 2 causes crosstalk from high wavelength bins to low wavelength bins. This crosstalk can be limited to a few percent with optimized thin film designs. The throughput of our fiber arrays is reduced by core-to-clad coupling and core-to-core concentricity at each coated fiber tip. These effects limit the number of useful coated tips, and result in an overall throughput of from 20% to 70%. Although our design can accommodate 10 to 15 bins, the data in this article was generated with either three or six bins as called out in the figure captions.

Fig. 2

Spectral array of coated fiber tips.

The key advantage of this design is speed—a spectrum is acquired during the dwell time of each pixel in the spatial scan. Beyond speed, this new design has several other advantages. First, wavelength bin centers and band-pass widths can be arranged with varying widths and spacings matched to a given application. The spectral widths and spacings in designs based on single bulk gratings are constrained by the diffraction equation. Second, the wavelength separation method has no influence on spatial scanning fidelity. A map of a given color will spatially register with a map of another color. Third, each polarization state is reflected in a similar manner. Fourth, the new design is compatible with confocal optics.

2.4.

Software and Firmware

The optical system described above is controlled by software running on a 1.73-GHz quad-core Intel i7-820 processor, and firmware running on a custom electronics box (see system photo in Fig. 3). The software, written in LabView, generates the graphical user interface (GUI) shown in Fig. 4. The firmware runs on board-mounted field programmable gate array and digital signal processor chips within the electronics box, which also provides several interconnects to the optical package and the quad box. These interconnects include the following inputs [start trigger, universal serial bus (USB), fiber to PMT], and outputs (scan mirror control signals, laser trigger, PMT analog output, A-to-D start trigger, sampling clock). A 14-bit digitizer card capable of 100 MS/s is mounted within the quad box chassis. Several events occur during operation: the GUI issues a start pulse, the electronics box sends control signals to the laser and mirrors, and the digitizer card receives three signals (start pulse, PMT output, and sampling clock). Finally, the quad box receives a serial array of acquisitions from the digitizer, and casts the data into a three-dimensional data cube.

Fig. 3

System photo: monitor (A), optical package (B), electronics box (C), quad core controller (D).

Fig. 4

System GUI showing controls, bin histogram, and multispectral image of mixed q-dots ($\text{bin}1=\text{laser}$, $\text{bin}2=500$ to 530 nm, $\text{bin}3=530-555\mathrm{nm}$, $\text{bin}4=555$ to 580 nm, $\text{bin}5=580$ to 600 nm, $\text{bin}6=600$ to 650 nm, and $\text{bin}7=650$ to 780 nm) ($\text{red}=\text{bin}7$, $\text{green}=\text{bin}6$, $\text{blue}=\text{bin}3$).

Selected slices of the data cube and a bin histogram are displayed on the GUI in real time. A screen shot from a scan on mixed quantum dots (q-dots) is shown in Fig. 4. The GUI includes settings for allocations amongst space, wavelength, and over-sampling, which are forwarded via the USB to the electronics box. Default settings are: 200 pixels per line, 150 lines per data cube, 15 spectral bins per pixel, and 10 samples per bin. As indicated in Fig. 5, the GUI allows multiple image display modes: any selected spectral bin in gray levels (A), the sum of all selected spectral bins in gray levels (B), three selected spectral bins assigned to display RGB levels (C), and any combinations of bins can be entered into formulae to generate a customized display. These formulae, for example, can assign gray levels to the delta between two bins normalized by a third bin, or a similar computation customized for each of the red, green, and blue levels of a display. An example is given by the image in Fig. 4 where bin 7 is assigned to red, bin 6 to green, and bin 3 to blue. The histogram at the top of Fig. 4 is a spectrum averaged over a frame (the wavelengths for each bin are defined in the figure caption). The two histograms at the right side of Fig. 4 are keyed to individual pixels in the image. Each scanned field generates 30,000 spectra. Our goal is to use multiple bins to colorize an image that clearly shows the presence of abnormal tissue at cellular spatial resolution. The GUI also provides regions-of-interest analysis tools, and storage of a sequence of frames from bin 1 to bin $N$ to facilitate development of the colorization formulae.

Fig. 5

Modes for extracting two-dimensional images from three-dimensional data cubes (all images occur at $10/\mathrm{s}$ and can be summed or saved as a series of frames over $t_0$ to $t_N$).

One useful colorization formula is based on the spectral angle mapping (SAM) algorithm¹⁵ given in Eq. (1). The SAM method computes the cosine of the angle between the signal ($S$) and reference ($R$) vectors by computing the dot product of the two vectors, and dividing by the product of their absolute magnitudes. In our case, each component of $S$ is represented by ${\mathrm{s}}_i$, laser-induced fluorescence from an unknown sample in the $i$’th spectral bin (for example ${\lambda }_N$ in Fig. 1). Each component of $R$ is represented by $r_i$, laser-induced fluorescence from a known reference in the $i$’th spectral bin. The software and firmware described above can determine the SAM angle for each pixel in a confocal scan, and colorize the resulting image according to the magnitude of the angle—all in real time. When the cosine is unity, the $S$ and $R$ vectors are perfectly matched. A key advantage of SAM images is that variations in brightness (due to fluorescent intensity or the degree of focus) do not impact the image. This is because SAM images are based on the angle between the two vectors as opposed to the magnitude of the vectors. Useful SAM images require appropriate reference vectors. As an example, we developed reference vectors for three q-dots that emit in blue, green, and red wavelengths. Figure 6 shows three scans of the green emitting q-dots using the three reference vectors. These images were generated by assigning pixel brightness to the intensity of one spectral bin, while assigning pixel color according to the value of $\cos \theta$ and the colorization code given in Fig. 6. As expected, the green q-dot emission is colorized by the green q-dot reference, but not by the blue and red references

Fig. 6

Spectral angle mapping (SAM) image with blue (a), green (b), and red (c) q-dot reference vectors ($200\times 150\mu \text{m}$).

2.5.

Sample Preparation Methods

Spatial resolution tests were performed with the USAF 1951 high resolution test target. This target contains bars with spatial frequencies ranging from 1 to $645.08\text{cycles}/\mathrm{mm}$ (finest bars are 0.8-μm wide). Three-micron diameter fluorescein-doped fluorescent beads were also used to assess image quality. These beads were drop cast from solution onto glass slides and dried in air.

Spectral resolution tests were performed with quantum dots (q-dots) that respond to blue excitation and exhibit fairly discrete emission spectra. Q-dots were selected for emission at 650 nm (CdSe/ZnS powder), 558 nm (CdSe/ZnS powder), and 486 nm (CdSe in toluene). The dots were suspended in toluene and drop cast onto glass slides.

Initial biomedical assessments were performed on fixed rat mammary tissue obtained from the Cedars-Sinai Medical Center. These samples¹⁶ include healthy left breast as well as right breast transfected with human breast cancer cells.

Ex-vivo images of nude mouse organs were obtained at the Cedars-Sinai Medical Center. These scans were acquired within minutes of euthanizing the animal to minimize changes in metabolic fluorophores. This mouse was 20-weeks old and was anesthetized/euthanized as approved by the institutional animal protocol (CSMC: IACUC#4397)

Additional ex-vivo studies have been undertaken on preclinical tumor xenografts generated by subcutaneous injection with human breast cancer cells (MDA-MB-231). Typically 2.5-million cells were injected into the flank of a 12-week old athymic nude female mouse and the tumor growth was monitored for 6 weeks until the tumor grew to about $1{\mathrm{cm}}^3$ in size. At this point, the mouse was anesthetized and euthanized as approved by the institutional animal protocol (CSMC:IACUC#4397). A large tumor with adjacent skin (see Fig. 7) was excised at the Cedars-Sinai Medical Center in Los Angeles, packed in Hypothermosol (available from BioLife Solutions Inc., Bothell, Washington), and shipped to Vermont for analysis. Storing and or shipping in Hypothermosol can preserve metabolic fluorophores for 2 to 7 days.

Fig. 7

Mouse with tumor on left flank [(a) before surgery, (b) during surgery].