1.

Introduction

Ovarian cancer has the lowest survival rates of the gynecologic cancers. This is mainly because it is predominantly diagnosed in stages III and IV due to the lack of reliable symptoms as well as the lack of efficacious screening techniques.¹ Currently, there is no single test for ovarian cancer detection. The combination of serum marker CA125 screening, transvaginal ultrasound (US), and pelvic examination yields a low-positive predictive value.^2–5 X-ray computed tomography scans have low sensitivities and are poor in detecting smaller metastasized tumors of less than 2-cm in diameter.⁶ Magnetic resonance imaging is costly and is typically used as a secondary imaging method when ultrasonography diagnosis is not sufficient.⁷ Positron emission tomography has limited value in lesion localization in the early stages of ovarian cancer.⁸

Within the near-infrared (NIR) optical window (700- to 900-nm wavelength), scattering is the most dominant light–tissue interaction, while the optical absorption coefficient of water is relatively low compared with the main absorber—blood hemoglobin. The transparency of water makes NIR light ideal to probe biological tissues up to several centimeters in depth using optical methods such as photoacoustic tomography (PAT), diffuse optical tomography (DOT), and spectroscopy (DOS).^9–11 PAT is an emerging modality that provides optical absorption contrast of targets or lesions at US resolution.^10,12–16 Transient acoustic waves generated from thermoelastic expansion, resulting from a transient temperature rise upon laser illumination, are then measured by an US array transducer and used to reconstruct the optical absorption distribution, which is directly related to the microvessel density of tumors or tumor angiogenesis.^17,18 The penetration depth of PAT is scalable with the US frequency, provided the signal-to-noise ratio (SNR) is adequate.¹⁰ In the diagnostic frequency range of 3 to 8 MHz, the penetration depth of PAT in tissue can reach up to 4 to 5 cm using NIR light,¹² which is compatible with the penetration depth of conventional transvaginal US. Though the resolution of PAT is excellent compared with a pure optical imaging modality, such as DOT, it lacks signal inside homogeneous structures and detects signals mainly from the surface due to US transducer bandwidth limitations.¹⁰ This makes the coregistration of PAT with conventional US inevitable for better mapping of light absorption to the corresponding anatomical structure. Based on statistics, up to 85 to 90% of ovarian cancers are epithelial ovarian carcinomas located at the outer surface of the ovary.¹⁹ Thus, the coregistered US/PAT that maps out the surface vascular density with the surrounding anatomical structure of the ovary is highly valuable in assisting physicians for early detection and characterization of ovarian cancers.

Several coregistered US/PAT systems have been developed,^20–26 but all of them lack real-time coregistration capability. Visual Sonics (Toronto, Ontario, Canada) provides coregistered high-frequency US and photoacoustic imaging products. Unfortunately, their frame rate is only 4 frames per second (fps) and the penetration depth is limited to about 1 cm. Recently, WinProbe Inc. (North Palm Beach, Florida) has developed both research and clinical imaging systems. Their research system is capable of providing coregistered US and PAT images in real-time with lower frame rates.²⁷ To facilitate the clinical use of the coregistered US/PAT modality, real-time coregistration of US and PAT is desirable to correlate the optical contrast with the anatomical structure for quick localization of suspicious areas. Achieving real-time coregistration of the two modalities is not an easy task since the data acquisition, storage, and processing of these two modalities are different at the technical level. The processor/controller for these tasks should have reconfigurability with negligible latency to perform switching between the two modalities and to achieve interlaced real-time imaging. A field programmable gate array (FPGA)-based reconfigurable processor was designed and implemented by our group to perform the front-end data acquisition and data preprocessing (US beamforming), for which a modular system design was implemented for lowest latency and best SNR.²⁸ In our previous system, only one coregistered frame per second was achieved using a 100-MHz Ethernet linked to a personal computer (PC). In this paper, a significantly improved system is reported in which a peripheral component interconnect express (PCIe) bus for data transfer is implemented. The PC program and FPGA code are optimized according to the new system structure, and multithreads techniques and Fourier domain fast data processing are implemented. These improvements allow us to achieve 15 coregistered US/PAT fps, and this speed is only limited by the 15-Hz laser pulse repetition rate (PRR). Additionally, the number of US channels is upgraded from 64 to 128, which allows us to scan a larger field of view for up to 120 deg. A custom-built probe consisting of a 128-channel transvaginal US array transducer, an optical fiber assembly, and a three-dimensional-printed housing is specially designed for the coregistration of two modalities in this system.²⁹

This paper will present the system architecture and optimal methods employed in the FPGA and PC software. Phantom experiments are conducted to evaluate the lateral resolution and axial sensitivity of the system. An example of real-time scanning obtained from an in vivo tumor-bearing mouse is presented to demonstrate the real-time imaging capability of the system. Examples of ex vivo malignant and normal ovaries are given to demonstrate the synergy of the coregistered US/PAT in identifying tumor vasculature and tumor morphology. To the best of our knowledge, the coregistered US/PAT imaging system reported in this paper is the first real-time system capable of scanning a larger field of view for up to 120 deg and 8 cm in depth with a user selectable configuration for either the US mode, photoacoustic imaging mode, or coregistered imaging mode.

2.

System Design

The first generation US/PAT system reported in Ref. 28 used an Ethernet cable for data transfer and a digital signal processor (DSP, TI6455) for data coordination. The speed of the system was 1 frame per second and the number of channels was 64. The DSP was used to form the pulse-echo transmit (Tx) timing first-in-first-out (FIFO) and to perform beamforming after fetching analog-to-digital converter (ADC) data from module FPGAs. The interface between DSP and module FGPAs is an external memory interface of the DSP chip, and the data transfer rate was low. Another bottleneck was the low-data transfer rate of the Ethernet cable (limited to 100 Mbps). Thus, the low-data transfer rate among module FPGAs, DSP, and PC limited the imaging frame rate to 1 frame per second.

To solve the low-data transfer rate problem, we have used PCIe and low-voltage differential signaling (LVDS) tracks in our current design. PCIe (up to 2.5 Gbps) has been used for data transfer between PC and Virtex 5 FPGA inside the PCIe board, which has served as a data coordinator. The LVDS between the PCIe board and module FPGAs has solved the problem of communications among different FPGAs. As a result, a 15 fps imaging frame rate has been achieved which is only limited by our laser repetition frequency.

2.1.

System Hardware and Data Acquisition Structure

The real-time system is designed to work in three different imaging modes: US mode, PAT mode, and coregistered mode. It requires different modules to perform data acquisition, transfer, and processing with negligible switch latency between the different imaging modes. From a hardware perspective, the real-time system mainly consists of a front-end analog circuitry, 8-module FPGAs, a PCIe card (Xpress LX110T by PLDA, San Jose, California), and a PC (Fig. 1). The front-end analog circuitry and 8-module FPGAs work together to perform US transmission focusing, data sampling, and US receiving sub-beamforming. The PCIe card works as a data coordinator. It obtains transmission and receiving beamforming delay profiles, calculated according to user’s imaging setting (transmission focus, number of scan lines, and angular scan range) from the PC. It then distributes the delay coefficients to the memories of each module corresponding to its attached 16 channels. Once sub-beamforming is done in each module FPGA, the FPGA on the PCIe card continues US beamforming and envelope detection. The resulting beam envelopes are transferred to the PC for scan conversion and display. For the PAT mode or coregistered mode, the PCIe card directly transfers the PAT radio frequency (RF) data to the PC. The PC software performs PAT delay-and-sum beamforming,^30,31 envelope detection, and scan conversion. Furthermore, the PC software graphics user interface allows user interaction for data storage and dynamic range selection for display with different color scales according to each imaging mode.

Fig. 1

The overall block diagram of the coregistered pulse-echo ultrasound (US)/PAT system. Each 16-channel module consists of five boards: a custom-built 16-channel Tx pulsars and Rx preamplifier board, two commercial 8-channel US-receiving boards, a Virtex-4 FPGA board for digital capturing and reconfigurable subprocessing, and a customized adapter PCB board. The adapter PCB connects the ADC signals of the two receiving boards to one module FPGA board and also connects the Tx control signals from the module FPGA to the Tx pulsars. Eight modules are plugged into the motherboard, which fans out all the 128 channels from the modules to the US probe connector.

The real-time system features 128 parallel channels, each of which has its own transmit/receive (Tx/Rx) circuitry along with ADC sampling at 40 MHz. It is a modular design (Fig. 1), where the front-end circuits of each set of 16 channels are grouped into one module that is plugged into a slot in the system motherboard.³² The transmission of every channel is controlled by the module FPGA using a Tx timing FIFO memory. Furthermore, ADC data are also captured by the module FPGA and stored in another Rx FIFO memory. US sub-beamforming of every 16-channel group is done by each module FPGA that features 2500 beam lines per second. US sub-beamforming data as well as PAT RF data from each module are collected by the Virtex-5 FPGA-based PCIe card plugged into the PC. All module FPGAs are connected to the PCIe card through another customized adapter board that uses LVDS tracks to achieve the ultrafast transfer rate. Here, each module FPGA has two pairs of dedicated one-way LVDS tracks single-edge pumped at 100 MHz and features a transfer speed of $200\mathrm{Mbit}/\mathrm{s}$ per module.³³ The modules are connected to the adapter board, and the adapter board is connected to the PCIe card. The PCIe card forms the final US beam from the sub-beamforming data, then stores it in an $8k\times 64\mathrm{bit}$ FIFO. After that, envelope detection is done by taking the absolute value followed by low-pass filtering. PAT RF data are captured and stored in a $32k\times 64\mathrm{bit}$ FIFO inside the PCIe card. The speed of communication between the PCIe card and the PC can reach up to $2.5\mathrm{Gb}/\mathrm{s}$, which adds a negligible latency to the overall acquisition time.

2.2.

Module FPGA Operation and Reconfiguration

Each module FPGA has two embedded quad-data-rate static random access memory (SRAM) chips which can be freely accessed by the PC through general purpose register input/outputs (GPIO) of the PCIe card. In the initialization stage, the PC transfers the time delay profiles for US transmission focusing and receiving beamforming to module FPGA’s SRAMs via the PCIe card. In every acquisition cycle, the PC program informs the PCIe card of the index of the active scan line through GPIOs. Each module FPGA temporarily stores the Tx and Rx coefficients of the active scan line in its internal cache memory (Fig. 2). The Tx coefficients of the active scan line contain the starting time for pulsing to each element, where a logical algorithm is used to construct the timing profile of every pulse for every element and store it to the Tx timing FIFOs.

Fig. 2

The block diagram of the internal structure of the module FPGA processor.

US sub-beamforming is done within each module FPGA by fetching the data from all 16 channels of the Rx FIFOs/RAMs while realigning and adding (delay and sum). A 50-MHz clock is used for beamforming, which allows forming the beam for one scan line in 102.4 μs for 4096 Rx FIFO points (sampled at 40 MHz, allowing a 7.7 cm maximum imaging depth for US pulse-echo). However, since the internal cache of the FPGA has to be filled with the Tx and Rx coefficients, and the formed beam has to be sent to the PCIe card, the total time needed for the beam envelope data to reach the PC is $400\mu \mathrm{s}$ or $2.5k\mathrm{lines}/\mathrm{s}$. For an image with 100 scan lines, after adding the scan conversion latency, the best frame rate achieved for US-only imaging is 20 fps.

In the PAT mode or coregistered mode, the DAQ must be synchronized with the trigger from the laser system. The trigger wire is attached to the PCIe card which keeps checking the status of the trigger signal. The PC program waits for this trigger and sets the imaging mode register to the PAT mode. The FPGA structure can be switched from the PAT mode to the US mode in a fraction of a microsecond. Both PAT and US data acquisition is done between the two laser pulses, which is valuable in that it keeps the US image free from laser lamp coherent noise. With this optimization scheme, 15 fps in both PAT and coregistered modes have been achieved and this frame rate is only limited by our laser repetition frequency.

2.4.

Imaging Probe

The imaging probe consists of a commercial 128 elements transvaginal US array (6-MHz central frequency and 80% bandwidth, W.L. Gore and Associates, Inc., Newark, Delaware) and 36 $200\text{-}\mu \mathrm{m}$ core fibers from a pair of high-power fiber-optic beam splitter assemblies (Fig. 3).²⁹ Both transducer and fibers are housed in a home-made protecting frame, which also keeps them from getting too close to the tissue and causing injury during imaging. Additionally, the fibers are positioned so that their distal ends are approximately 1 cm away from the tissue surface during imaging. In this way, the beam is expanded at the tissue surface and the fluence is decreased even more. The 1-cm distance also ensures that the light propagates from the fibers, with a numerical aperture of 0.2, to the tissue surface without reflecting off the surfaces of the plastic cover and transducer. The illumination patterns in tissue at different imaging depths have been reported in our prior publications.^29,35 Studies have shown that the laser beams from the fibers spread out as they propagate through tissue and merge together at a depth of about 1 cm to form a fairly uniform illumination pattern at the center directly below the US imaging probe. Since the vagina muscle wall is about 8-mm to 1-cm thick, a uniform illumination of the ovary behind the vaginal wall can be achieved. The light fluence delivered by the probe on the tissue surface is about $17\mathrm{mJ}/{\mathrm{cm}}^2$, which is below the American National Standards Institute maximum permissible exposure (MPE) of $24\mathrm{mJ}/{\mathrm{cm}}^2$ at 750 nm. Light from the laser is first coupled into the input ends of the two splitters using a convex lens and 50/50 beam splitter arrangement. The laser is a tunable-wavelength Ti:Sapphire (Symphotics TII, LS-2211, Camarillo, California) optically pumped by a Q-switched Nd:YAG (Symphotics TII, LS-2134). It delivers 20-ns pulses at a 15-Hz repetition rate with 20 mJ/pulse at the imaging wavelength of 750 nm.

Fig. 3

Light delivery system and experimental setup of in vivo tumor-bearing mouse.

3.

Results

To obtain the lateral resolution of the system, a $100\text{-}\mu \mathrm{m}$ diameter black thread was placed in a water tank at a 30-mm depth with its orientation perpendicular to the imaging plane. The lateral profiles of pulse-echo US and PAT were measured and are shown in Fig. 4(a). Additionally, the maximum amplitude of the beam envelope in the PAT mode was recorded and the system noise was estimated from the root mean square value of the beam envelope when laser transmission was disabled. SNR or sensitivity of the black thread was calculated by $20*{\log }_{10}$(max PAT envelope/system noise). By varying the distance between the black thread and the probe surface from 1 to 6.25 cm with an interval of 2.5 mm, the SNR at each depth is calculated and the curve is shown in Fig. 4(b). The SNR dropped about 24 dB from 1 to 6 cm.

Fig. 4

(a) Lateral profiles of pulse-echo US and PAT measured from a $100\text{-}\mu \mathrm{m}$ black thread. (b) Axial sensitivity or SNR of PAT mode.

Since there is no theory for wideband transducer arrays, narrowband theory is used for the lateral resolution estimation.^36,37 For a pulse-echo mode, the narrowband theory indicates that the beam profile follows a ${\mathrm{sinc}}^2(Dx/\lambda z)$ pattern in the lateral direction, where $D$ is the effective size of the transducer in the lateral direction $x$, $\lambda$ is the US wavelength, and $z$ is the depth. In the PAT mode, the lateral beam profile is given by a sinc function because of one-way reception only. Note that the 65 nearest elements were used in the beamforming, which resulted in an effective transducer array size $D$ of 9.56 mm. The measured lateral resolutions of both pulse-echo and PAT modes agree well with theoretical values. From theory, full widths at half maximums of lateral beam profiles are 1.37 deg in pulse-echo and 1.86 deg in the PAT mode, while the measured values are 1.56 and 2.01 deg, respectively.

An imaging setup for the in vivo animal experiment is shown in Fig. 3. A calibrated intralipid of $0.03{\mathrm{cm}}^{-1}$ absorption coefficient and $3{\mathrm{cm}}^{-1}$ reduced scattering coefficient was placed in a plastic bag and used as a coupling medium. The tumor-bearing mouse was under anesthesia by inhalation of 1.5% isoflurane. The animal protocol was approved by the Institutional Animal Care and Use Committee of the University of Connecticut. US imaging was performed first, and the real-time images (Video 1) were used to locate the tumor area at 20 fps. Once the tumor area was identified, the coregistered US/PAT scan was performed. Video 2 shows the coregistered images and demonstrates the real-time imaging capability of the system while scanning the tumor area at 5 fps (averaging PAT acquisitions of three laser pulses). To guide the readers, we have included white circles in both movies to mark the tumor area. For better visualization of the tumor vasculature content, all PAT images in Video 2 were normalized to the peak of the PAT beam envelopes of all frames during the scan procedure and are shown with a 15-dB dynamic range. The US images were shown with a 40-dB dynamic range. The wavelength used was 750 nm. In the second part of the experiment, the computer-controlled tunable Ti:Sapphire laser was used to obtain 10 PAT acquisitions at each wavelength from 745 to 835 nm with a 10-nm step size. After that, the coregistered US and oxy-saturation (${\mathrm{sO}}_2$) image was formed, where the ${\mathrm{sO}}_2$ value was estimated from the relative ratio of oxy-hemoglobin to the total-hemoglobin concentrations, pixel-by-pixel using a gradient descent fitting technique to minimize the least squares error between the measurement at different wavelengths and a linear model comprising the concentration of oxy- and deoxy-hemoglobins.²⁸ Note that the measurements are constructed from the PAT value at each pixel normalized by the laser energy per pulse as measured at each wavelength. Also, the defined model assumes that the fluence profile over the imaging plane does not change over the selected wavelength window. For extensive details about the model used and assumed approximations, readers are referred to Ref. 28. Figure 5 shows the fitting results of one pixel of 99.14% ${\mathrm{sO}}_2$. The probe output optical energy was measured during data acquisition at each wavelength and used to compensate the PAT measurements before ${\mathrm{sO}}_2$ estimation. Figure 6 shows the coregistered US/PAT images at 745, 805, and 835 nm wavelengths along with the resulting coregistered ${\mathrm{sO}}_2$ image estimated from all 10 PAT wavelengths. Note that the tumor closer to the tissue surface appears to have a lower ${\mathrm{sO}}_2$ as compared to the inside; this observation is quite similar to what we observed in a previous study.²⁸ Skin, in general, has higher absorption at lower wavelengths due to a higher deoxygenated hemoglobin content and thus lower ${\mathrm{sO}}_2$.

Video 1

Real-time ultrasound scan for quick tumor area identification of tumor-bearing mouse (QuickTime, 8.7 MB) [URI: http://dx.doi.org/10.1117/1.JBO.19.7.076020.1].

Video 2

Real-time coregistered US/PAT scan of mouse tumor area in 5 fps (QuickTime, 3.9 MB) [URI: http://dx.doi.org/10.1117/1.JBO.19.7.076020.2].

Fig. 5

Fitting of 10 PAT measurements for one pixel to obtain a smoothed image of the oxygen saturation.

Fig. 6

The first three images from left are coregistered US/PAT at different wavelengths (indicated in the upper left corners), the first image from right is a coregistered $\mathrm{US}/{\mathrm{sO}}_2$ image obtained using gradient descent estimation from 10 PAT acquisitions at 10 different wavelengths. US images are in grayscale and the tumor is shown as a round hypoechoic region marked by an arrow.The PAT absorption images (first three) and ${\mathrm{sO}}_2$ image are shown in color. The right color bar is the ${\mathrm{sO}}_2$ scale.

In the third set of experiments, a malignant and a normal ovary were imaged ex vivo. Each ovary was mounted on a supporting frame made of a transparent thin optical fiber net and was submerged inside a calibrated intralipid tank of $0.03{\mathrm{cm}}^{-1}$ absorption coefficient and $4.0{\mathrm{cm}}^{-1}$ reduced scattering coefficient. The ovary was located about 10-mm deep from the probe surface. The PAT signals of the ovaries were weaker than that of the black thread, thus an average of 64 laser pulses was used to improve the quality of images which reduced the PAT image rate to about 4 s per frame. Figure 7 shows the coregistered US/PAT images which reveal the blood contents on the surface of the malignant and normal ovaries. By comparing both images, we can see much a higher vasculature content on the surface of the malignant ovary than that of the normal one.

Fig. 7

Examples of coregistered US/PAT images of a (a) malignant ovary and (b) a normal ovary. US images are shown in gray scale, and the level of intensity increases from black to white. The PAT images are shown in color, and the absorption increases from red to yellow. Note that the images in this set are shown with 40-dB dynamic range in pulse echo US and 15-dB dynamic range in PAT. The PAT images were normalized to the maximum envelope of the malignant ovary.

4.

Summary

A new US/PAT system with real-time imaging capability to assist physicians in transvaginal scans for ovarian cancer detection and diagnosis is presented. With the ultrafast PCIe data transfer, implementation of multithreads and Fourier domain data processing in PC interface software, and dedicated FPGA structure and optimization, the system can achieve 20 fps in US mode, 15 fps in PAT mode, and 15 fps in the coregistered mode. The system frame rate is limited by the laser PRR, along with the number of PAT averaging times needed to improve the SNR. System lateral resolution matches well with narrow-band estimation in both US pulse echo and PAT mode. The real-time scan capability of the system has been shown using an in vivo tumor-bearing mouse. Furthermore, the image quality achieved by the system was demonstrated by imaging ex vivo human ovaries submerged in a scattering medium. We envision that the real-time US mode will be used for fast scans to quickly identify ovaries and the coregistered US/PAT will be used to characterize the ovaries and assist physicians with diagnosis.

Currently, a larger number of PAT acquisitions are averaged to obtain high-quality images, showing features of the vasculature content of the ovaries. These features in PAT images can be used to further validate the image classifier developed by our group.^38–40 It is possible to obtain acceptable images even without averaging at 15 fps. However, when the signal is too weak such as that obtained from large ovarian cancers of more than 8 to 10 cm in size, a lower dynamic range has to be used and most features of the images are lost. For in vivo imaging, it is expected that a higher vascular content may be obtained than that of ex vivo scan due to blood loss of the excised ovaries. Of course, the vagina muscle wall and the surrounding tissue may scatter and absorb light which may cause a reduction in the PAT signals. Currently, we are in the process of obtaining an approval for human subject protocol, and we will report our in vivo studies using the reported coregistered US/PAT system in the near future.