Biomedical imaging continues to advance our capability to detect cancer and investigate associated biological events. Anatomical imaging with computed tomography (CT), magnetic resonance imaging (MRI), and ultrasound remain the primary imaging modalities in oncologic imaging. Functional and molecular imaging with radioactive contrast agents can provide whole-body information about tumor metabolism, proliferation, and hypoxia relative to healthy tissues. These modalities use noninvasive imaging to guide treatment selection along with diagnostic pathology of biopsy. However, relatively low sensitivity of anatomic modalities and poor spatial resolution of molecular imaging limit the ability of noninvasive imaging modalities for guiding surgery, which is the primary treatment option in many cancers. Optical imaging with colored or fluorescent contrast agents complements noninvasive imaging by providing high resolution and real-time visualization during endoscopic and surgical procedures.

Intraoperative fluorescence imaging provides real-time visualization of signal from fluorescent reporters over a large field of view (FOV) with exceptional sensitivity and resolution.^1,2 Fluorescence imaging is used during surgery to assess patency of blood vessels and ureters.³ Near-infrared (NIR) fluorescence (700 to 900 nm) is often preferred for deep tissue imaging due to its higher depth of penetration and lower background fluorescence.^1,4 However, NIR fluorescence is invisible to the human eye, and emitted light levels are typically low for the human eye to see, requiring camera-based detection. Clinical optical imaging systems typically consist of highly sensitive, scientific digital cameras with appropriate illumination source and optical filters for fluorescence detection.^1,4,5 Fluorescence image information is acquired via attached computer, processed to reduce background signal and enhance contrast, then displayed on an adjacent digital monitor alongside or overlaying the reference brightfield image [Fig. 1(a)]. We hypothesized that surface projection of fluorescence information on the surgical field [Fig. 1(b)] will improve the detection and removal of tumor tissue.

Fig. 1

(a) Cartoons of state-of-the-art fluorescence-guided surgery in which fluorescence information is displayed on a flat-screen monitor and (b) optical projection of acquired luminescence (OPAL) fluorescence-guided surgery in which fluorescence information is directly projected onto the surgical field, “highlighting” the tissues with high concentrations of fluorescent contrast agent (green).

Digital light processing (DLP) technology enables fast, high resolution, and high contrast digital image projection. DLP systems utilize digital micromirror arrays to project bright and high resolution color images over large areas.⁶ Due to the increase in computer speed and economics of scale, DLP technology has been adapted for use during medical procedures.^7,8 Herein we report a novel DLP-based strategy, optical projection of acquired luminescence (OPAL), for automated fluorescence imaging and direct super-imposition onto the surgical field.

We constructed a small prototype system to demonstrate proof-of-principle for OPAL applications in oncologic procedures. A white light-emitting diode (LED) light source (KL200 LED, Leica, Buffalo Grove, Illinois) provided illumination continuously during all procedures. Excitation light for fluorescence was provided by high power 760-nm LED (M780L2 and LEDD1B, Thorlabs, Maryland) with $769\pm 20.5\mathrm{nm}$ band-pass filter (FF01-769/41, Semrock, Rochester, New York). A 0.33-MP monochrome complementary metal-oxide semiconductor (CMOS) camera (Firefly MV FMVU-03MTM-CS, Point Grey Research, Canada) with 16-mm fixed focal lens [17HD (2/3” 16 mm F/1,4 C-Mount, Tamron, Japan)] and 785-nm long-pass edge filter (BLP01-785R, Semrock) was used for fluorescence detection. A 20 lumen, $608\times 684\text{pixel}$ resolution pico projector kit (DLP Lightcrafter™, Texas Instruments, Dallas, Texas) was used to project images.

The OPAL system was positioned vertically, 30 cm above the operating field. This position gave a $3.5\times 5{\mathrm{cm}}^2$ camera FOV for the camera (about $13\text{pixels}/\mathrm{mm}$) and $7.5\times 5.5{\mathrm{cm}}^2$ (about $10\text{pixels}/\mathrm{mm}$) for the projector. Image acquisition, processing, and projection were controlled using MATLAB (The Mathworks, Inc., Natick, Massachusetts). The acquisition speed of the camera was set to 30 Hz within the FlyCapture 2.4 software (Pointgrey) and images acquired through MATLAB Image Acquisition toolbox. Monochrome images were passed from MATLAB to the Lightcrafter stored memory via universal serial bus connection using the MATLAB Instrument Control toolbox. Acquired fluorescence images were aligned within the projector FOV and projected as a monochrome green image onto the operating field.

After verification of image alignment using fluorescent phantom materials, in vivo studies were performed to evaluate utility in oncologic procedures. All procedures involving animals were conducted in accordance with protocols approved by the Washington University Animal Studies Committee. Mice were anesthetized with isoflurane (2% v/v in 100% oxygen) and maintained at a surgical plane for imaging and invasive procedures. Animals were euthanized at the end of procedures while anesthetized. Use of OPAL during experiments was documented using a consumer-grade video camera (Panasonic HC-V100 1.5 MP camcorder).

For sentinel lymph node mapping, a 12-week old female FVB mouse was anesthetized with isoflurane. The hair covering its left front limb was removed by gentle clipping and cream depilatory. The mouse was then positioned on its right side with the left side exposed to the OPAL FOV. A 20-μl volume of 60-μM indocyanine green (ICG) in phosphate buffered saline (PBS) was injected intradermally into the left footpad. Fluorescence images were projected as 8-bit monochrome green images onto the surface of the mouse. The transport of ICG from the site of injection to a regional lymph node was visualized at $0.3\text{frames}/\mathrm{s}$. Fluorescence signal migrated from the site of injection, culminating in a bright green spot in the axillary region appeared over 10 to 15 s (Fig. 2). Exposure of the lymph node via skin incision resulted in intensification of the bright green projection and dissection of the lymph node. Identification of the lymph node was confirmed by gross and histologic inspection.

Fig. 2

Projection of raw fluorescence image information onto the surface of a mouse after intradermal injection of indocyanine green into the paw (*). Lymphatic transport to the regional lymph nodes was directly visualized by bright green light.

To demonstrate use for cytoreduction in oncologic surgery, the OPAL was used to guide identification and removal of tumors in a model of peritoneal breast cancer metastasis. Luciferase-transfected mouse mammary carcinoma (4T1luc) tumor cells were injected intra-peritoneally in one 8-week old female balb/c mouse. The mouse was injected intravenously with IntegriSense 750 (Perkin Elmer, Waltham, Massachusetts) which targets ${\alpha }_{\nu }{\beta }_3$-integrin receptor, commonly overexpressed in tumor tissue.⁹ The mouse was anesthetized with isoflurane and hair was removed on and around the abdomen and pelvic region. The mouse was placed on the OPAL imaging platform in dorsal recumbency. To perform dynamic projection using the OPAL system, the captured image was denoised using a $5\times 5\text{pixel}$ ($0.5\times 0.5\mathrm{mm}$) median filter and thresholded to give optimal visual contrast between tumor (high intensity) and nontumor (low intensity) regions.

Prior to surgery, two large areas of fluorescence signal were observed as projected green light on the abdomen and projected onto the operating field as green color, easily visualized concurrent with white LED epi-illumination (Fig. 3). A midline incision was made to expose the internal organs. A relatively large $3\times 5\mathrm{mm}$ tumor was first illuminated and removed. The second fluorescent region was identified as a nodular mass extending from the pylorus of the stomach posteriorly along the duodenum that was not evident on visual inspection alone. Further exploration of the abdomen revealed high fluorescence from the bifurcation of the uterus and cervix. No other suspicious areas were identified by OPAL or by visual inspection. Identification of the first and second excised tissues as tumor was confirmed by histology. Tumor cells were not identified in the uterus, but integrin expression can be upregulated in these tissues due cyclic hormonal changes.¹⁰

Fig. 3

Frames from a video capture of OPAL used to identify and remove peritoneal tumors 24 h after injection of tumor-selective near-infrared (NIR) fluorescent molecular probe, IntegriSense 750. Detected NIR fluorescence was projected onto the surgical field, identifying tumors with bright green light.

The OPAL system provided visual contrast from NIR fluorescent contrast agent accumulation in regional lymph node and tumor tissue with minimal alteration of the normal surgical environment. Bleed-through of projected light was not detected during our testing of this prototype, but this may become a concern with use of a higher power projector.

The OPAL strategy will be best used in open surgeries that provide a relatively flat FOV and do not require significant magnification. These procedures include sentinel lymph node biopsy⁴ and cytoreduction in breast and ovarian cancer.^11,12 Fluorescence imaging during endoscopic procedures, including robotic surgeries, requires attention to a digital display and would not benefit from direct image projection. Although NIR imaging has the potential for greater depth detection relative to visible wavelengths, fluorescence imaging systems based on a planar reflectance geometry have maximum reported depth of penetration of 4 cm and typically $<2\mathrm{cm}$.^1,4 Our system leverages these advantages and combines them with DLP technology to achieve real-time reprojection of NIR fluorescence images correctly aligned with anatomical structures.

Further work is in progress to improve the sensitivity of fluorescence detection and optimize the contrast enhancement during surgical procedures necessary for clinical applications. This small prototype was sufficient for preclinical studies, but the working distance must be increased to at least 1 m to prevent obstruction during surgeries. A high speed, scientific grade camera will be required for sensitive fluorescence detection at this distance, similar to those used in previously reported intraoperative fluorescence imaging systems.^1,11 In addition, image correction and background removal algorithms may also be included to optimize signal detection and image contrast.^1,4 Application-specific software and hardware-based image processing¹³ are needed to improve the speed to real-time of $30\text{frames}/\mathrm{s}$ or greater and add quantitative capabilities.

Visual contrast enhancement is dependent on the intensity of projected light relative to the surgical illumination and tissue of interest. In these initial experiments, we chose green light because the human eye is most sensitive to this color and green is relatively absent in the surgical field. The DLP projector used in this prototype is rated at 20 lumen light output, which is insufficient for large FOV and high ambient lighting during surgery. Full-size DLP projectors are now capable of greater than 10,000 lumen light output. Newer laser-based DLP projectors promise even brighter projection.⁶

And lastly, there are no fluorescent molecular probes approved for use in humans in the United States at this time. ICG is the only approved NIR fluorescent reporter⁴ although targeted agents are in various stages of the translational process.^12,14,15 As NIR fluorescent molecular probes become approved for diagnostic use in human medicine, fluorescence imaging will become an essential tool for oncologic surgery.

Fluorescence molecular imaging methods and targeted contrast agents are progressing steadily toward use in oncologic surgical procedures. Clinical imaging systems are already in place for detection of fluorescence contrast during endoscopy and open surgery. The fluorescence image projection method demonstrated through this prototype OPAL system represents a promising method for utilizing fluorescence molecular imaging to improve outcomes in oncologic surgery.