Review Papers

Review of spectral imaging technology in biomedical engineering: achievements and challenges

[+] Author Affiliations
Qingli Li

East China Normal University, Key Laboratory of Polar Materials and Devices, Shanghai 200241, China

Columbia University, Department of Psychiatry, Brain Imaging Lab, New York 10032

Xiaofu He, Dongrong Xu

Columbia University, Department of Psychiatry, Brain Imaging Lab, New York 10032

Yiting Wang

East China Normal University, Institutes for Advanced Interdisciplinary Research, Shanghai 200062, China

Hongying Liu, Fangmin Guo

East China Normal University, Key Laboratory of Polar Materials and Devices, Shanghai 200241, China

J. Biomed. Opt. 18(10), 100901 (Oct 10, 2013). doi:10.1117/1.JBO.18.10.100901
History: Received January 21, 2013; Revised September 6, 2013; Accepted September 10, 2013
Text Size: A A A

Open Access Open Access

Abstract.  Spectral imaging is a technology that integrates conventional imaging and spectroscopy to get both spatial and spectral information from an object. Although this technology was originally developed for remote sensing, it has been extended to the biomedical engineering field as a powerful analytical tool for biological and biomedical research. This review introduces the basics of spectral imaging, imaging methods, current equipment, and recent advances in biomedical applications. The performance and analytical capabilities of spectral imaging systems for biological and biomedical imaging are discussed. In particular, the current achievements and limitations of this technology in biomedical engineering are presented. The benefits and development trends of biomedical spectral imaging are highlighted to provide the reader with an insight into the current technological advances and its potential for biomedical research.

Figures in this Article

Spectral imaging is also known as imaging spectroscopy, which refers to the technology that integrates conventional imaging and spectroscopy methods to obtain both spatial and spectral information of an object. It was originally defined by Goetz in the late 1980s and discussed for remote sensing of the Earth.1 Spectral imaging can be divided into multispectral imaging, hyperspectral imaging (HSI), and ultraspectral imaging according to its spectral resolution, number of bands, width, and contiguousness of bands. Multispectral imaging systems generally collect data in few and relatively noncontiguous wide spectral bands, typically measured in micrometers or tens of micrometers. These spectral bands are selected to collect intensity in specifically defined parts of the spectrum and optimized for certain categories of information most evident in those bands. While HSI systems can collect hundreds of spectral bands, ultraspectral imaging systems collect even more. Figure 1 shows the concept of the hypercube data captured by a spectral imaging system. The spectral imaging data can be visualized as a three-dimensional (3-D) cube or a stack of multiple two-dimensional (2-D) images because of its intrinsic structure, in which the cube face is a function of the spatial coordinates and the depth is a function of wavelength. One of the important advantages of this technique is that it can acquire reflectance, absorption, or fluorescence spectrum for each pixel in the image, which can be used to detect the biochemical changes of objects that cannot be identified with traditional gray or color imaging methods. Spectral imaging technology has been originally substantiated within remote sensing fields, such as airborne surveillance or satellite imaging, and has been successfully applied to mining and geology, agriculture, military, environmental, and global change research.2

Graphic Jump LocationF1 :

The concept of spectral data cube. The data cube contains two spatial dimensions (x and y) and one spectral dimension, in which the cube face is a function of the spatial coordinates and the depth is a function of wavelength.

According to the electromagnetic theory, different biochemical constituents commonly have different spectral signatures.3 These signatures are usually generated by the interactions between materials and electromagnetic waves, such as electron transition, atomic and molecular vibration or rotation. The biological and pathological changes in tissues and organs also have a close relationship with the spectra. Spectral characteristics in different wavelength regions yield a distinguishable spectral signature, making pathological changes distinguishable. Therefore, the spectral imaging technology also can be extended to the biomedical engineering field to estimate the physiological status of biological tissues, since it can take advantage of the spatial relationships among the different spectra in a neighborhood. This technology opens new prospects for life science by which scientists can identify and quantify the relationships among biologically active molecules, observe living organisms noninvasively, perform histopathological and fluorescent analyses, and enhance biological understanding of diseases. In the past decades, researchers have developed various spectral imaging systems for the biochemical analysis of various biological organs and tissues. These studies have shown that the extension of this technology to the biomedical engineering field has incredible potential for researchers to get more information on organs, tissues, and even cells than those traditional optical imaging methods (such as CCD cameras and light microscopes). Biomedical spectral imaging technology has attracted more and more attention and has gained importance in research today.

This paper summarized the origins and major developments of spectral imaging technology used in the biomedical field. The development of biomedical spectral imaging systems, comparison of different spectral imaging methods, system performance, in vivo, fluorescence, and histological analyses, as well as current limitations and challenges of this technology are discussed. The purpose of this paper is to highlight the advantages and disadvantages of biomedical spectral imaging reported previously and to identify fundamental and applied research issues that occurred in the current and future biomedical applications.

Most traditional biomedical optical imaging methods can only capture gray or color images of biological samples. The targets of interest in these kinds of images are generally analyzed by their spatial properties such as size, shape, and texture. It has been widely recognized that the monochrome and RGB color imaging methods have limitations in the early detection and identification of tissue abnormalities.4,5 The obtained diagnostic information is poor, since in most cases, the metabolic or compositional alterations occurred during the progress of the disease and do not significantly alter the color characteristics of abnormal tissues.6 Another generally used optical method is the spectroscopic diagnostic technology, which can obtain an entire spectrum of a single tissue site within a wavelength region of interest. This method is often referred to as the point measurement method, which cannot provide the spatial information of samples. Different from those traditional optical diagnostic methods, biomedical spectral imaging technology can capture the contiguous spectrum for each image pixel over a selected wavelength interval. This character makes it possible not only to detect some physiological changes of biological tissues by their reflectance or transmittance spectral signatures, but also for early diagnosis of some diseases since the shapes of the spectra yield information about biological samples. Some advantages of biomedical spectral imaging (multispectral, hyperspectral) technology over conventional monochrome, RGB, and spectroscopy are presented in Table 1. From the table, it can be seen that the biomedical spectral images contain more information than the traditional monochrome, RGB, and spectroscopy methods. Biomedical spectral images make it possible to take advantage of the spatial relationships among the different spectra in a neighborhood, which allows more elaborate spectral-spatial models for a more accurate segmentation and classification of the image.7 Therefore, the spectral imaging technology can find potential applications in pathology, cytogenetics, histology, immunohistology, and clinical diagnosis.

Table Grahic Jump Location
Table 1Comparison of monochrome, RGB, spectroscopy, multispectral, and hyperspectral features.

In the past decades, different kinds of spectral imaging methods and related technology have been proposed in acquiring the spectral image data of natural materials. To make the content and structure clear, this review mainly focuses on four typical approaches: the whiskbroom, pushbroom, staring, and snapshot, which have been commonly used in the remote sensing field and now have been extended to the biomedical imaging applications. Other related ones, such as the Raman, the miscellaneous, and Hadamard methods, will not be discussed in detail in this review.

Whiskbroom

The whiskbroom imaging mode, also known as the point-scanning method, was used originally by the Earth Resources Technology Satellites and then implemented by the Airborne Visible/Infrared Imaging Spectrometer. In the whiskbroom sensor, rotating mirrors were often used to scan the landscape from side to side perpendicular to the direction of the sensor platform. The rotating mirrors redirect the reflected light to a point where a single or just a few sensor detectors are grouped together. As shown in Fig. 2(a), a single point is scanned along two spatial dimensions (x and y) by moving either the sample or the detector. Then the reflected light is dispersed by a prism and recorded by a linear array detector. The spectral image data cubes (x, y, λ) can be acquired by scanning over the 2-D scene (x, y) and by dispersing in the wavelength domain (λ). In the whiskbroom mode, two-axis motorized positioning tables are usually needed to finish the image acquisition, which makes the hardware configuration complex. In addition, this kind of imaging mode is also usually time-consuming because it needs to scan both in the x and y spatial dimensions. Therefore, another scanning approach with high acquisition speed and high sensitivity has been proposed, that is, the spectral confocal laser-scanning method. This method is usually accomplished by illuminating and acquiring images through confocal or conjugate pinholes to restrict the in-focus optical section thickness and by incorporating multiple lasers and wavelength dispersive spectrophotometers to get spectral information. An advantage afforded by this method is the ability to control depth of field, eliminate or reduce background information away from the focal plane, and the capability to collect serial optical sections from thick specimens. Another advantage of the whiskbroom imaging is that it has fewer sensor detectors to keep calibrated as compared with other types of sensors. Most commercial spectral confocal scanning instruments belong to this kind of imaging mode.

Graphic Jump LocationF2 :

Typical spectral imaging approaches. (a) Whiskbroom. (b) Pushbroom. (c) Staring. (d) Snapshot.

Pushbroom

The pushbroom approach, also known as line scanning, has been used by some earth observation satellite systems operating from space, such as the SPOT system and the Advanced Land Imager (ALI). Different from the whiskbroom method that scans one point at a time, the pushbroom method can simultaneously acquire a slit of spatial information, as well as spectral information corresponding to each spatial point in the slit for one scan, that is, a special yλ image with one spatial dimension (y) and one spectral dimension (λ) can be taken at the same time with a matrix detector. As shown in Fig. 2(b), the pushbroom method collects a slit image from the object dispersed onto a 2-D detector, in which the spatial information is displayed along one axis and wavelength information along the other. Then the spectral image data cube can be obtained by scanning the slit in the direction of another spatial axis. A pushbroom scanner can get more light than the whiskbroom one, since it can stay at a particular area for a longer time providing a long exposure on the matrix detector and a relatively high spectral resolution as well.8 However, in most of the pushbroom imagers, either the camera or the object should move accordingly as one line of the image is acquired at each scan and the move should be synchronized with the frame acquisition rate of the array detector in order to ensure the measurement of a smooth image. Another approach to perform the pushbroom procedure is to put a movable slit aperture or a rolling shutter into the conjugate image plane to eliminate out-of-focus light. This method allows for higher illumination transmission while exhibiting only a slight increase in the confocal depth response, which has been widely used in multispectral line confocal imaging systems.9

Staring

The staring approach (also known as band sequential method) is a spectral scanning method that acquires a single band 2-D gray-scale image (x, y) with full spatial information at once. As shown in Fig. 2(c), this mode generally uses filters [such as filter wheels containing fixed bandpass filters, linear variable filters (LVF or wedge filters), variable interference filter (VIF), and tunable filters] instead of a grating or prism in front of a matrix detector. The light passes through the focusing optics. Then it is filtered, which induces only a narrowband segment of the spectrum that impinges on the focal plane of the detector (typically is a matrix CCD). Therefore, the 2-D image in one wavelength can be captured at a given time, and the image cube is filled by tuning the output wavelength of the filter as a function of time.10 In the staring mode, the entire scene is imaged onto the focal plane, one spectral band at a time, and goes through the spectral wavelengths, which is different from the whiskbroom and pushbroom approaches. Therefore, the number of spectral bands acquired is completely up to the user. Also, and importantly, a dynamic range can be preserved, since different exposures can be employed at different wavelengths.

Snapshot

The snapshot (also known as single-shot) method is intended to record both spatial and spectral information on an area detector with one exposure. Unlike the whiskbroom, pushbroom, and staring modes that need to scan either in the spatial dimension or in the spectral dimension, thereby limiting their temporal resolution, the snapshot mode is an imaging technique with no need to scan at all. As shown in Fig. 2(d), the snapshot mode can acquire a complete spectral data cube in a single integration by directly imaging the remapped and dispersed image zones onto a CCD detector simultaneously.11 Although this mode can acquire data directly with minimal postprocessing to build a 3-D data cube, its spatial and spectral resolutions are limited as the total number of voxels cannot exceed the total number of pixels on the CCD camera. Therefore, for a given CCD, one can always increase the spatial sampling at the expense of spectral sampling, and vice versa.12

Comparison of Different Approaches

Among these four kinds of spectral imaging approaches, each one had both advantages and disadvantages when they were applied to analyze biological tissues. Table 2 shows the comparison of the whiskbroom, pushbroom, staring, and snapshot imaging modes. There is no “absolutely” best mode, but in order to select the most suitable one for a given biomedical application, the characters of different biological tissues and the detection objective must be considered.

Table Grahic Jump Location
Table 2Comparisons of whiskbroom, pushbroom, staring, and snapshot approaches.
Table Footer NoteaSome whiskbroom and pushbroom systems allow selection of spectral range and resolution such as Zeiss and Nikon systems listed in Table 4.
Table Footer NotebRefers to the time spent capturing the same size of data cube (x, y, λ).
Table Footer NotecSome laser scanning systems (for example, the Zeiss and Nikon point scanning systems) have as fast a speed as the snapshot systems. Some systems with conventional mechanical x-y scanning will take a longer time to capture a data cube (x, y, λ).
Table Footer NotedSystems based on liquid crystal tunable filter (LCTF) usually take a longer time than the laser scanning systems because the switching speed is limited by the relaxation time of the crystal. Acousto-optical tunable filter (AOTF) can switch wavelengths <1ms but might be limited by photon count.

From Table 2, it can be seen that the whiskbroom and pushbroom modes use a dispersive element [prism, grating, prism-grating-prism (PGP), etc.] to split the light, a factor that gives an advantage of uniformly high efficiency, low scatter, and low cost to these kinds of imagers. Another advantage of the pushbroom mode is that the collected images reflect only the yλ plane, making a single line of light feasible as the source for image illumination.13 This approach is suitable for applications of searching an object with specific spectral signatures as the entire spectrum of each pixel in the slit image is available in real time. In addition, systems that are based on dispersion elements, such as prisms and gratings, are also efficient for measuring the spectral images through confocal microscopes, which implement the scanning procedure by scanning optics.14 The whiskbroom and pushbroom modes also have disadvantages. The optical design is complex and the data cube collection time is long for conventional mechanical scanning systems. Although some optical scanning systems such as the Zeiss and Nikon laser scanning systems can capture data as fast as the snapshot systems, the acquisition of high-contrast images commonly accompanies strong photobleaching, which is of particular concern in prolonged imaging experiments.15 In general, the whiskbroom and pushbroom imagers were often coupled with microscopes and used for fluorescent and histopathological analyses.

The most important advantages of the staring mode are the short collection time of data cube and the ease of coupling with other medical optical instruments, as no relative spatial move is required between the sample and the detector. Another advantage is the flexibility of selecting the optimal spectral range for each application. These advantages make it suitable for a wide range of biomedical applications, such as detecting organs, tissues, and cells in vivo or in vitro when it is coupled with a camera lens, endoscope, or microscope.13,16 There are also some disadvantages of using this mode, such as a lower spectral resolution, lower transmission rate (typically reaches only 35 to 60% depending on the type of device and model), and a higher cost than the whiskbroom and pushbroom modes (except the laser scanning systems). Although tunable filters attempt to shorten the total capture time by selectively acquiring spectral bands, or by altering each channel’s exposure time as a function of wavelength so that the total data capture time can be saved in bright channels, there are still shortcomings that can be difficult to overcome, such as the low throughput and sequential acquisition mode, particularly in multifluorophore imaging, which requires at least 30 spectral channels to be captured.15 The interferometer-based spectral imaging systems, however, usually have high optical throughput, high and variable spectral resolution, mechanical/temperature stability, and flexible selection of wavelength range. They still have limitations for practical applications, such as the varying sensitivity in the entire spectral region and the computation intensity for data transform.17

For the snapshot mode, no scanning in either spatial or spectral domains is needed for obtaining a spectral data cube, making it attractive for applications requiring fast spectral image acquisitions. Therefore, it is an ideal solution for applications where imaging time is a major factor. It is suitable for time-sensitive imaging experiments, such as the observation of fast-diffusing molecules or the determination of temporally resolved dynamic biological processes. It is also suitable for detecting oxygen saturation maps in the retina of human eyes when it is coupled with a fundus camera. This has been demonstrated by Johnson et al. with computed tomography imaging spectrometer (CTIS) with an exposure time of 3ms.18 However, this mode is still in the early stages and not fully developed. Only a few implementations that rely on complicated fore-optics design and computationally intensive postprocessing for image reconstructions are currently available, with limitations for range and resolution of spatial and spectral dimensions.17

In summary, each spectral imaging mode has advantages and disadvantages when used in biomedical analysis. For example, some whiskbroom and pushbroom systems can get high spectral and spatial resolution and are low costing, but their hardware assembly is usually complex. The staring mode can be assembled compactly and coupled easily with other optical instruments, but it is more expensive than the others. The snapshot mode can quickly capture spectral data cube, but its spectral and spatial resolutions are limited. However, the advantages of different modes can be exploited fully if they are selected correctly and used according to the objectives of different biomedical analyses. We will discuss this in detail in the following sections.

System Architecture

To acquire a 3-D data cube, spatial (2-D) and wavelength as a third dimension of a biological tissue, a spectral imaging system suitable for biomedical applications needs to be developed. As shown in Fig. 3, the hardware architecture of a typical biomedical spectral imaging system generally consists of four parts: collection optics or instruments, spectral dispersion element, detector, and system control and data collection module.

Graphic Jump LocationF3 :

The system architecture of a typical biomedical spectral imaging system. The reflected or transmitted light collected by the collection optics passes through the spectral dispersion element and is focused on the detector. The system control and data collection module can control the detector and the spectral dispersion element, capture the spectral images, and output results.

The collection optics or instruments part refers to the collection and imaging optics such as microscopes, endoscopes, camera lens, etc., which can produce an image on the spectral dispersion element. The spectral dispersion element is the heart of the system that enables the separation of the light into different wavelengths. Dispersive spectrometers and spectral filters are two kinds of commonly used wavelength dispersion devices. The dispersive spectrometer is a collection of transmitting or reflecting elements separated by a distance comparable to the wavelength of the light under investigation. It often refers to prisms, gratings, PGP, or beam splitters, which can be used with whiskbroom, pushbroom, and some snapshot imaging modes. Spectral filters have the characteristic to pass radiation through a very narrow bandpass, or spectral bin, and this can be spectrally tuned over a wide spectral range, usually in a very short time. Acousto-optical tunable filter (AOTF), liquid crystal tunable filter (LCTF), and circular and linear variable filters (CVF and LVF) are the most popularly used tunable filters in spectral scanning instruments. Another generally used spectral dispersion element is the interferometer, which can obtain spectral images by changing optical path-length difference (OPD) between two beams, recording the interference intensity as a function of the OPD (interferogram), and Fourier transforming the interferograms pixel by pixel.19 For example, the Michelson interferometer and the Sagnac interferometer have been used to develop some spectral imaging systems for different biomedical research.20,21 The detector is used to acquire the light intensity required to measure the spectrum at each pixel in the image. Some sensors such as the photomultiplier tube, linear CCD, area CCD and CMOS, focal plane array, and electron multiplying charge coupled device (EMCCD) can be used as a detector according to different imaging requirements. In recent years, the scientific CMOS (sCMOS) has been introduced and also used in biomedical imaging as it offers an advanced set of performance features, such as high resolution, without compromising read noise, dynamic range, or frame rate. These features make the sCMOS ideal for high-fidelity, quantitative scientific measurement. Therefore, some commercial sCMOS [for example, the ORCA-Flash2.8 and Flash4.0 camera (Hamamatsu), Andor and PCO series sensors, and Rolera™ Bolt Camera (QImaging)] can also be used as a detector of spectral imaging system.

In the past decades, researchers have developed various spectral imaging systems for biochemical analysis on various biological organs, tissues, and cells. Table 3 shows some selected spectral imaging systems used in the biomedical field. We will break up the discussion of instrument configuration according to the imaging mode in the following subsections.

Table Grahic Jump Location
Table 3Current biomedical spectral imaging systems.
Current Biomedical Spectral Imaging Systems
Whiskbroom and pushbroom systems

Whiskbroom (point-scanning) and pushbroom (line-scanning) biomedical spectral imagers generally use the spatial scanning configurations with prisms, gratings, or beam splitters to create spectral discrimination. These kinds of imagers are popular for remote sensing applications as an airplane or satellite flying over the surface of the Earth provides a natural scanning procedure. The biomedical applications of whiskbroom and pushbroom systems are mainly used in microscopic forms because the scanning procedure can be carried out by moving of the microscope stage, and the acquisition time is usually not a problem for some microscopic observations. In the past decades, various microscopic multispectral or HSI systems have been developed and used in biomedical engineering. Huebschman et al. developed a prototype HSI microscope by coupling a grating-based SpectraPro 556i spectrograph (Acton Research Corp., Acton, MA) to a microscope. The system allows measurement of wavelengths approximately from 400 to 780 nm by a 1536×1024-pixels Photometrics-Synsys 1600 air-cooled CCD camera (Photometrics Inc., Tuczon, AZ) with the stage incrementally moving in 0.1-mm or larger intervals.13 This is the earliest report about the development of the pushbroom microscopic HSI system. Then they analyzed the characteristics and capabilities of this kind of system and tested it with a range of samples relevant to cytogenetic, histologic, fluorescence in situ hybridization (FISH), cell fusion, and tumor cells in their recent studies.4244 Some commercialized PGP-based imaging spectrograph also can be used to design a HSI microscope. For example, Li et al. have developed the microscopic pushbroom HSI system by coupling an ImSpector imaging spectrograph (ImSpector series) produced by Spectral Imaging Ltd. (Oulu, Finland) to a microscope.4749 Some researchers also developed a short-wave infrared hyperspectral pushbroom imaging system for quality control of herbal medicines.126 Another kind of spatial scanning systems, the spectral confocal microscopes, also have been developed and widely used in biological analysis. The spectral confocal microscopes were originally developed in different laboratories as application-specific instruments in early 1990s.127,128 Since then, a number of point-scanning and line-scanning spectral confocal microscopes have been introduced that expand upon these capabilities by allowing the acquisition and unmixing of data from multiple fluorescent probes with overlapping fluorescence emission spectra.129131 Some commercial spectral confocal microscopes also have been provided by different manufacturers such as Leica (TCS SP), Olympus (FluoView), Nikon (C1si), and Zeiss (LSM). Although each of them has their own spectral detector design for detection of wavelength-dependent signals, three different types of spectral units, that is, dispersive prism, diffractive grating, and multichannel array, are usually used by these instruments. Recently, some new types of spectral confocal microscopes have been proposed, such as the wide spectral range confocal microscope,132 the multispectral line confocal imaging microscope,9 the confocal-spectral microscope,133 and the multichannel spectral imaging laser scanning confocal microscope.134 These types of equipment have the capacity to obtain unique fluorescence spectra of specific dye molecules that will increase our understanding of the biological fluorescence emission process. Also, some other similar microscopic whiskbroom or pushbroom systems have been presented in recent studies, such as the hyperspectral fluorescence lifetime imaging microscope,45 the Hadamard transform spectral microscope,31 the hyperspectral microscopic system,51 the prototype open-frame MACROscope,52,53 etc. Besides microscopes, some other optical devices such as colposcope, ophthalmological lens, endoscope, and camera lens also have been coupled to the monochromator-based imaging spectrograph to form different kinds of whiskbroom or pushbroom biomedical spectral imaging systems.24,34,36,37,41,54,55,135,136 In addition, some spectral confocal scanning laser ophthalmoscopes also have been developed for retinal imaging.137,138 One of the advantages of the pushbroom and whiskbroom imagers is that they can scan the target in one or two spatial dimensions, which makes the imaging range of these two modes larger than the staring and snapshot ones. Therefore, the spatial scanning imager is suitable for coupling with a microarray scanner to analyze spotted microarrays. The pushbroom hyperspectral microarray scanner has the advantage of scanning an entire microarray at high spectral and spatial resolutions, allowing the identification and elimination of unwanted artifacts. It also improves the accuracy of microarray experiments greatly.13,25,26 There are also some disadvantages with traditional spatial scan imagers—low light throughput due to slit caused vignetting, nonuniform efficiency, being at maximum at the blaze wavelength, limited spectral range due to the grating order superposition, etc. In addition, some grating-based monochromators coupled with microscopes will require very bright light sources in order to compensate for the low throughput of the grating and complicated calibration methods.78

Staring biomedical spectral imaging systems

The staring biomedical spectral imagers use spectral filters (e.g., filter wheels containing fixed bandpass filters, AOTF, and LCTF) to acquire a single-band 2-D gray image with full spatial information and form a spectral data cube by scanning in the spectral domain through a number of wavelengths. The most basic implementation of staring spectral imaging is the use of a rotatable disk called a filter wheel carrying a set of discrete bandpass filters. Afromowitz et al. have developed a color filter wheel–based multispectral imaging system to evaluate burn depth on human palms and feet.112 This is the earliest study on the staring biomedical spectral imager and its applications. In recent studies, similar systems have been developed for quantitative analysis of the human tooth and ovary.111,113 The advantage of these filter wheel–based imagers is that they are easy to use and relatively inexpensive. However, they also have some disadvantages, such as slow wavelength switching, narrow spectral range and low resolution, mechanical vibration from moving parts, and image misregistration due to the filter movement. Therefore, some other spectral filters need to be used in order to overcome these limitations. There are mainly two filters widely used in the staring biomedical spectral imager: AOTF and LCTF. AOTF is a rapid wavelength scanning solid-state device that operates as a tunable optical bandpass filter based on light–sound interactions in a crystal. The major function of AOTF is to isolate a single wavelength of light from a broadband source in response to an applied acoustic field. The main advantages of AOTF are good transmission efficiency, fast scan times, large spectral range, and flexible controllability and programmability. AOTF is also attached easily to a microscope, endoscope, and camera lens. Schaeberle and Treado, et al. have developed multispectral and HSI microscopy based on AOTF.139,140 Then, Wachman et al. developed an AOTF microscope to map hemoglobin saturation and oxygen tension in mouse brains.99,100 Since then, more AOTF-based microscopy HSI systems have been proposed and used for different biological analysis, for example, the HSI system,86 the molecular HSI (MHSI) system,96 the HSI microscope,141 etc. Besides microscope, endoscope and camera lens also have been attached to AOTF to detect biological tissues in vivo. For example, some AOTF-based endoscope spectral imaging systems have been developed and used to detect skin tumors, thorax surgeries, and cancerous tissues.5,16,84,85,87,88 Furthermore, some camera lens with standard interface can be coupled to an AOTF adapter directly and used for in vivo imaging.8082 The advantages of AOTF-based imagers include quiet and vibration-free operation, high switching speed, spectral selectivity, spectral purity, and flexibility. On the other hand, LCTF is a solid-state instrument that utilizes electronically controlled liquid crystal cells to transmit light with a specific wavelength and the elimination of all other wavelengths. The LCTF approach also has been used in coupling with the microscope,65,8992 fundus microscope,114,117119 and camera lens5658,60,62,64,6870,77 to create spectral image cubes for biological imaging in vivo and in vitro. Although the electronic controller of the LCTF is able to shift the narrow bandpass region throughout the entire wavelength range covered by the filter unit, the wavelength switching speed depends on the relaxation time of the liquid crystal as well as the number of stages in the filter. Typically, it takes tens of milliseconds to switch from one wavelength to another, which is far longer than the response time of the AOTF.17 The commercial AOTF and LCTF products also have been widely used to acquire biomedical spectral images in the visible (VIS) and near-infrared (NIR) spectral regions. For example, some researchers have developed different kinds of systems based on the AOTF component (Brimrose, Sparks, MD),5 the AOTF imaging modules (ChromoDynamics, FL),99,141,142 and the CRi LCTF (Cambridge Research and Instrumentation, Woburn, MA).91,92 In addition, some commercial instruments have also been manufactured based on these components, such as the HSi-300 and HSi-440C HSI systems (Gooch & Housego, FL), the Nuance,™ and Vectra™ multiplexing microscopes (CRi). Compared to the fixed interference filters used in the filter wheels, AOTF and LCTF can be flexibly controlled by a computer. They do not have moving parts and, therefore, do not suffer from the problems associated with the rotating filter wheels, such as speed limitation, mechanical vibration, and image misregistration. However, the AOTF- and LCTF-based imagers are in all cases expensive and have delicate and low throughput.12 Therefore, an interferometer also has been used for spectral imaging as it usually has high optical throughput, high and variable spectral resolution. In general, it is possible for Michelson interferometer and Sagnac interferometer to acquire 2-D spatial and one-dimensional spectral information of objects by applying a Fourier transform spectrometer algorithm and the Van Cittert-Zernike theorem. For example, Inoue et al. discussed the performance of Fourier-transform spectral imaging based on the Michelson interferometer in early 1990s. Potter et al. developed an FTIR system by coupling mid-IR Michelson-type step-scan interferometer to an IR microscope.143 Magotra et al. developed an hyperspectral biomedical imager imaging system using a spatially modulated Fourier transform interferometric device. This device is based on a Sagnac interferometer, which creates an interferogram that contains the desired spectral information.144 Fisher et al. obtained spectral images of a tooth by combining imaging optics, a Sagnac interferometer, and an imaging detector.145 Besides the Michelson and Sagnac interferometers, researchers found that Fizeau interferometer, based on double Fourier interferometry, can also be used for spectral imaging.146 In recent years, various interferometric spectral imaging systems and sensors have been developed and used for biomedical analysis, such as the spectral karyotyping (SKY), dynamic monitoring of biomolecular interactions, and live epithelial cancer cells analysis.20,109,147,148 Besides the interferometer, some other devices such as the rotating direct vision prism,149,150 VIF,78 CVF, LVF,42,151 and the dispersion of optical rotation152 can also be used to form biomedical staring imagers. Although the staring biomedical spectral imagers do not need to scan in the spatial dimension, most of them still need to scan in the spectral dimension. The scanning mechanism decreases their temporal resolution and limits their potential use in some real-time imaging. However, the staring spectral imaging systems are still the most popular ones in current biomedical applications.

Snapshot biomedical multispectral imaging systems

A snapshot biomedical multispectral imaging system can acquire both spectral and spatial information in a single exposure. This kind of imaging system can capture a scene of multispectral images faster than most of the spatial scanning and spectral scanning imagers. In recent years, researchers have proposed some methods for biomedical snapshot spectral imaging, such as the CTIS,120 image mapping spectrometer,125 coded aperture snapshot spectral imaging,153,154 image replicating imaging spectrometer (IRIS),114116 imaging slicing spectrometer,12 and Fourier transform–based snapshot imaging systems.155,156 A major advantage of this kind of imager is speed. The faster speed makes it suitable for studies in endoscopy detection, fluorescent probes analysis, retina oxygen saturation measurement, and rapid processes that other spectral imaging instruments can hardly handle. This kind of imager has some other advantages, among them time-resolved spectral imaging of transient phenomena possibility, high reliability and robustness as a result of not having moving parts, high signal throughput as no multiplex losses, etc.116 However, the dimensions of the spectral data cube obtained by snapshot imagers depend on the size of the CCD sensor they used, as the total number of voxels cannot exceed the total number of pixels on the CCD camera. Therefore, the spatial resolution and spectral resolution trade-offs of the system need to be considered according to different biomedical applications. Although improvements are needed to address the issue of spatial or spectral resolution, the snapshot system can capture hypercubes without any scanning that represents a new trend in instrument development for biomedical spectral imaging techniques.

Commercial systems

In addition to the different kinds of experimental spectral imaging systems developed by researchers in laboratories, some commercial biomedical spectral imaging systems have also been produced in recent years. Table 4 lists some commercial biomedical spectral imaging systems manufactured by different companies. Most of these systems were developed based on the pushbroom and staring imaging mode and were designed to meet a wide range of biomedical applications. Some systems were also designed for a particular application that can perform in-depth analysis on a specific tissue or disease with custom software. These commercial systems generally have modular hardware and friendly man–machine interface software, which make them more reliable than those experimental systems developed in laboratories, and these systems are suitable for doctors and medical researchers with little technical background and expertise. For example, the DySIS™ (Dynamic Spectral Imaging System, DySISmedical, UK) has been used to discriminate high- from low-grade cervical lesions and non-neoplastic tissues.157,158 The Nuance™ Multispectral Imaging Systems (CRi) has been used to assess blood flow and oxygenation in implantable engineered tissues,159 and to evaluate morphological distributions of miRNAs and their putative targets in thin tissue sections with the inForm™ image analysis software.160 However, most of the commercial softwares generally provide basic spectral image processing and analysis functions, and can hardly meet some particular research requirements. Therefore, the secondary software development is often needed by the users to implement their new analysis algorithms. The Gooch & Housego, Inc. considered this problem and has offered an open-source application software platform for imaging and control of automated microscopes on multiple operating systems (Windows, Mac, and Linux). Surface Optics Inc. also provides the HyperSpect™ cross-platform operating software, which can export data with a format supported by the third-part software such as ENVI or MATLAB. This characteristic is very important for the commercial systems in promoting the research implementations in different biomedical areas.

Table Grahic Jump Location
Table 4Some commercial biomedical spectral imaging systems.
System Performance

It is important to evaluate the performance of a biomedical spectral imaging system since different applications usually require different performance parameters. The main performance parameters of a biomedical spectral imaging system generally include spatial resolution, spectral range, spectral resolution, dynamic range, sensitivity, etc. These parameters depend mainly on the capability of the optical system (the optical head, the spectral dispersion element) to produce an undistorted, unaberrated, and intense data cube of a scene, as well as the ability of the detector to convert an image into a digital form with low noise and fine spatial or spectral sampling.161 Some performance parameters of current biomedical spectral imaging systems have been listed in Table 3. A brief discussion of the most important performance parameters and their relationship with the applications follows.

Spatial resolution

In biomedical imaging, the spatial structure information is one of the important factors for histological analysis and disease diagnosis. The inclusion of spatial information can increase the biological information content derived from data acquired with spectral imaging systems. As shown in Fig. 4, different biological objects usually have different spatial scale requirements for detection. Spatial resolution of a spectral imaging system determines the closest distinguishable features in the objects. With a proper selection of the system configuration, useful spatial information and quantification can be achieved by sufficient biological sampling, as different spectral imaging methods usually have different spatial resolutions ranging from a few micrometers to a few centimeters. For example, cells expressing each color of fluorescent protein can be located and clusters of cells expressing two or three colors usually reinforce the need for increased spatial resolution (<1μm) to facilitate single cell detection.26 A spectral imaging system with spatial resolution of 0.45 mm and 17-cm-diameter field of view (FOV) is sufficient for hemoglobin oxygen saturation detection in sickle cell disease patients and in some other clinical investigations.70,73 Therefore, proper selection of spatial resolution is important for different kinds of biomedical applications.

Graphic Jump LocationF4 :

Schematic representation of spatial scale of biological objects. As different biological objects generally have different spatial scales ranging from a few micrometers to a few centimeters, a proper selection of spatial resolution is usually needed to detect the object properly.

Spatial resolution defines the level of spatial detail depicted in an image, which determines the closest distinguishable features in an object. The theoretical spatial resolution mainly depends on two factors: the wavelength (λ) and the numerical aperture (NA) of optics. In addition, the pixel size of the detector and the magnification also play a role because they determine the sampling frequency, which must be sufficiently high to achieve full resolution.162 The spatial resolution of most spectral imaging systems can dominantly deteriorate by astigmatism, which is an off-axis aberration that increases with the square of both the off-axis angle and NA.163 Significant loss in spatial resolution due to prism mount misalignment in a chromotomosynthetic HSI system has been analyzed by Bostick et al. They found that the relative effect is less in the shorter-wavelength region, which indicates that spatial performance is also affected by wavelength.149 However, astigmatism is not a problem in a spectral confocal laser scanning system because the laser submits single points from the FOV sequentially, not multiple points simultaneously. For a line-scanning spectral imaging system, the spatial resolution in the direction of the slit width is affected by the slit width, whereas the spatial resolution along the slit length is usually affected by the magnification and camera pixel size,44 while for a spectral filters-based imaging system, the spatial resolution is also related to the types of tunable filters (AOTF, LCTF, etc.) the system used.4,70,87 Currently, spectral images acquired by microscopic biomedical spectral imaging systems are usually in the order of 0.3 μm or finer, whereas those systems coupled with camera lens generally acquire higher spatial resolution data, usually in the order of 0.45 mm or finer.44,70

There is also another problem to consider in the bright-field light microscopy and fluorescence microscopy combined with spectral imaging function. That is, how to produce high spatial resolution or clear images of a focal plane deep within a thick tissue. For example, a fluorescent cell might be 5 to 15 μm thick, whereas the thickness of the imaging plane of a high-NA objective (NA1.3) is only 300nm or less. Thus, the vast majority of the cell volume is out of focus and becomes blurred.164 Optical sectioning may be one of the approaches used in solving this problem. It can get better background rejection, improved contrast, and more accurate spectral characterization by reducing out-of-focus artifacts. Compared to common imaging with wide field, optical sectioning generally led to greater than 3× improvement in the rejection of out-of-focus fluorescence emissions and 6× greater peak-to-background ratios in biological specimens, yielding better contrast and spectral characterization, which is important in unmixing signals in biological samples and suppressing a phototoxicity and bleaching of fluorophores.165 The commonly used methods for optical sectioning are confocal laser scanning microscopy and two-photon microscopy. These methods are powerful means to eliminate the background caused by out-of-focus light and scatter from images and are well matched to the needs of spectroscopy, providing both convenient formats and high optical throughput. Conchello et al. and Svoboda et al. have summarized these techniques for optical imaging of thick tissues.164,166 Some new optical sectioning techniques, such as planar illumination and structured illumination have been proposed in recent years. These methods are particularly good for applications involving high speeds, large FOVs, or long-term imaging, which have been reviewed by Mertz.167 Structured illumination is a way to make it possible to perform optical sectioning with a conventional microscope, which is generally radiated through a grid of horizontal lines and is then reflected off a beam splitting mirror. Then the image of the thick sample with both wide-field and optical section containing grid lines passes back through the beam splitter to the camera. Thus, the optically sectioned images can be reconstructed by some algorithms from a conventional wide-field microscope.168 Most of these techniques can also be used for spectral imaging of thick tissues. For example, Oruanaidh et al. proposed a Bayesian spectral analysis method to obtain optical sectioning using structured light imaging in a conventional wide-field microscope.169 Webb et al. presented a wide-field fluorescence lifetime imaging system with spectral resolution and optical sectioning to achieve five-dimensional fluorescence images of biological tissues.170 In addition, the digital micromirror device also has been employed as a spatial excitation light modulator for either spectral imaging or fluorescence lifetime imaging.171,172 In recent years, single-point optical sectioning systems for microspectroscopic analysis have been reported, some of which have been adapted for scanning, particularly for Raman spectroscopy.173 The optical sectioning spectroscopic systems using slit scanning have been applied both for transmission and Raman spectroscopic measurements also.174 The programmable array microscope based on programmable spatial light modulators, developed by Hanley et al., is another kind of optical sectioning microscope of particular interest for spectroscopy.165 These studies show that optical sectioning has improved spatial resolution in the intensity images by removing out-of-focus blur. With optical sectioning, a high spatial resolution image generally demands a greater signal-to-noise ratio (SNR) than a conventional intensity image does. This will make high S/N ratios imperative when applying the technique to thick biological tissues.175 Therefore, suitable specification of the optical system is generally needed to produce optimal trade-offs between reducing the speckle noise and preserving the optical sectioning to get clear images. Some of these studies also demonstrate that the spectral imaging technology combined with optical sectioning is essential for spectroscopic characterization. For example, the spectral signature changes of tissues are generally associated with their pathologic states. The ability to localize these changes within cells and tissues will improve the diagnostic capabilities of such method. In some cases, the localization of probe molecules within a specimen requires that they can be distinguished from sample autofluorescence or from out-of-focus probe-fluorescence. Therefore, it is important for the spectral imaging microscope to reduce both types of background through the use of optical sectioning methods to improve the detection and accurate localization of targets.165 However, there are no hard and fast prescriptions on which technique is most suitable for optical sectioning. The researchers must consider what factor is important in the biomedical analysis (such as speed, resolution, and FOV) as the optimization of one factor often leads to degradation of the others.167

Spectral range

Spectral range refers to the range of wavelengths over which the biomedical spectral imaging system can measure the spectrum. As shown in Tables 3 and 4, different biomedical spectral imaging systems generally work in different spectral ranges as the properties of the optics are different. The spectral range is usually given by the minimum and maximum wavelengths at which the spectral response exceeds some threshold (for example, 10%). Generally, it is determined by the kind of illumination source, optical head (such as microscope, endoscope, camera lens, etc.), spectral dispersion element, and detector that the system uses. First, the transmission of a microscope objective, endoscope optics, and camera lens will limit the wavelengths of the incident light.152 Second, different illumination sources and spectral dispersion elements can only work in different wavelength ranges. For instance, a single AOTF unit generally covers a specific wavelength range, as the components for constructing it have different characteristics, which can only accommodate a particular spectral region. To acquire spectral images that cover the spectral region from 450 to 2500 nm, three different commercial imaging AOTF modules (Gooch & Housego) working at different wavelength ranges (i.e., 450 to 800 nm, 900 to 1700 nm, and 1300 to 2500 nm) are needed. At last, the spectral response of the detector is also an important characteristic that determines the working wavelength range of the system. For inspection of biological tissues using spectral information in the VIS and short-wavelength NIR regions, the silicon-based CCD detectors have been used in reflectance and transmittance spectral imaging systems.4,44,111 For an NIR spectral region (900 to 2500 nm), the InGaAs image sensor, which is made of an alloy of indium arsenide (InAs) and gallium arsenide (GaAs), is usually used.82,113,176 With the exception of the CCD and InGaAs spectral range, system cost will be significant primarily due to the required customization. In the visible and near infrared (VNIR) spectral range, CMOS cameras can be used, but only for the low-end applications.

According to the electromagnetic theory, different biochemical constituents, biochemical changes, and labels generally have different spectral signatures in a certain wavelength range.177 Therefore, a proper spectral range selection is usually needed for a particular analysis purpose. For example, to map the oxygen saturations in living tissues, a spectral range (such as 500 to 800 nm) which can cover the spectral signatures of HbO2 and Hb can usually be used for in vivo imaging. For fluorescent analysis, a spectral range from 400 to 750 nm is usually selected as the excitation spectral signatures of most of the fluorescent probes are located in this wavelength region. Another wavelength region from 900 to 1700 nm is usually used for teeth diseases diagnosis, as dental enamel manifests high transparency for the light in the NIR spectral region. In general, flexibility in selecting the optimal spectral range for a particular application is important. We will discuss this in the following sections.

Spectral resolution

Spectral resolution describes the ability of a biomedical spectral imaging system to distinguish two signals that are separated by a small spectral difference. It can be represented by the full width at half maximum (FWHM), which can be checked by measuring the spread in the spectrum of a very narrow source.178 The spectral resolution can be easily determined in any instrument by using a light source that emits a spectrum with a pure monochromatic line. Spectral resolution of a spectral imaging system generally depends on the type of the spectral dispersion element that the system uses. As shown in Tables 3 and 4, most grating-based pushbroom spectral imaging systems have the higher spectral resolution, as it is mainly determined by the slit width and the optical aberration. For those AOTF-based spectral imaging systems, the spectral resolution is uneven across the whole wavelength range. Shorter wavelengths generally have higher resolutions than longer wavelengths.17 For example, the corresponding spectral resolutions of an AOTF adapter with spectral range from 550 to 1000 nm are generally in the range of 2 to 20 nm.84,96,179 For an LCTF-based system, the spectral resolution is typically of the order of several nanometers, although a narrower bandpass can also be constructed.180 In some applications, the spectral resolution that is needed is limited, or there is a need for only few wavelengths that are well known. For example, diagnostic imaging might be fine with tens of wavelengths over a visible range, but 10 signaling probes would require 30 wavelengths with 2 to 4 nm spectral resolution, according to the theoretical analysis and experiments proposed in Refs. 181 and 182. In such a case, it is possible to use a rather simple method that splits the few spectral bands of interest to the different regions in space, so that the whole information can be measured simultaneously. For instance, a snapshot imager usually can work in a broad wavelength range with high spectral resolution at the cost of scarifying the spatial resolution, yet it is suitable for retinal imaging and detecting fast metabolic processes.

Other performance

There are also some other performance parameters to consider in the spectral imaging system, such as photon collection efficiency, data acquisition time, dynamic range, etc. The photon collection efficiency can evaluate the light level that the spectral imaging system can achieve. The photon collection efficiency of a spectral imaging system is usually different for each wavelength as the illumination, the NA of objective, the spectral response of the detector, and the transmission of the optics are not uniform. The sources detected by the spectral imaging systems tend to be weak and spatially incoherent, which makes the photon collection efficiency very critical in biological imaging. However, some spatial or spectral scanning spectral images usually have low photon collection efficiency for incoherent sources, which leads to a very small absolute number of collected photons. For example, the pushbroom spectral imager images only one slit at a time and will reject the out-of-slit photons. Therefore, a given element in a scene of spectral image tends to contain very few photons and hence has a poor SNR.28 The emission spectra in fluorescence measurements are usually inherently noisy when compared to broadband measurements as the narrow-band filtering required to image the fluorescence emission at many discrete wavelengths naturally results in decreased signal strength.141 In response to this problem, researchers have proposed theoretical analysis and a number of different designs to maximize the light gathering efficiency of the systems. R. Neher and E. Neher indicated that there are theoretical problems for the highest possible photon collection efficiency originating from the fact that both emission and absorption spectra of simultaneously present dyes’ overlap and their signal quality is limited by photon shot noise, as well as by photobleaching. They also defined the figure of merit (FoM) to measure the efficiency of a given system or measurement configuration.181 Some high–photon collection efficiency or high-throughput spectral imaging systems also have been developed for working with the weak, incoherent sources common to biomedical applications.19,28,183 These studies show that there are generally some ways to compensate for the photon collection efficiency, such as the use of a higher–quantum efficiency detector and a high-power light source to increase the exposure time. However, long exposure time may not be appropriate in applications where the scene is changing on a time scale of the acquisition.180 The exposure time also influences the data acquisition time, another parameter that depends on the effective single band time and on the total number of bands that should be measured. One of the fair comparisons of different acquisition methods can be based on the data acquisition time in which each method obtains information in order to satisfy the requirement of the measurement. As shown in Table 3, the snapshot imaging mode usually has the shortest data acquisition time as it does not need to scan to capture multispectral images. However, the data acquisition time of the whiskbroom mode is generally longer than other modes since it needs to scan in both x and y dimensions. In general, the measurement of spectral images requires significantly more time than the time required in acquiring a single frame using a traditional optical imaging system since the spectral images must be measured at different wavelengths. The time limitation of spectral imaging technology can be critical: for example, the high throughput that is required in many biomedical applications, the photobleaching influence in fluorescence, and the phototoxicity or image motion in applications such as retinal imaging. The dynamic range of a spectral imaging system refers to the largest difference in the intensity of incident light that can be observed in one scene of the spectral image, which depends largely on the dynamic range of the detector the system uses. In addition, due to the temperature and illumination fluctuation, the performance of the spectral imaging system usually gets worse in a real working condition. There are some methods that can be used to improve the system performance. One method is to add collimating lenses in front of each detector to reduce the wavelength shift and improve SNR. Another one is to use a cooling system to stabilize the temperature fluctuation. However, these methods will make the system more complex and cumbersome.151 The balance between system performance and complexity should be considered when developing a biomedical spectral imaging system.

System Calibration and Data Preprocessing

After the spectral imaging system was developed, one of the important steps was to ensure that the system would provide an accurate measurement of the spectrum. That is, the spectral imaging systems need to be calibrated and corrected to produce a flat spectral response before further experimental measurements. These steps are important for ensuring reproducible results and for comparing spectra across multiple systems.141 One way to determine the wavelength accuracy of the spectral imaging system is to calibrate it with a low-pressure Hg+/Ar+ discharge lamp that covers the wavelength range from 365 to 850 nm. However, the pure Hg+/Ar+ lamp usually emits deep UV light that is dangerous to the exposed skin and can cause blindness to unprotected eyes, thus making the measurement difficult to perform. Therefore, an eye-safety, multiion discharge wavelength calibration lamp (MIDL) distributed by LightForm Inc. can be used for wavelength calibration from 365 to 912 nm simultaneously.163 Lerner and Zucker have proposed the MIDL calibration method in detail and have derived reference spectra from the MIDL data to accurately predict the spectral resolution, ratio of wavelength to wavelength, contrast, and the aliasing parameters of any confocal spectral imaging system.184 This method has also been used by Leavesley et al. to calibrate the HSi-300 HSI system (ChromoDynamics, Orlando, FL).141 In addition, some other spectrally well-known light sources, such as lasers (for example, 632.8 nm by HeNe lasers), fluorescent lamps, and broadband lamps equipped with interference bandpass filters, can also be used for wavelength calibration.17 Other calibration methods also have been presented recently. For example, three National Institute of Standards and Technology traceable Spectralon standards (Labsphere, North Sutton, NH) of different colors have been used to calibrate the multiaperture system, which can capture six identical images of the human fundus at six different spectral bands.122 Xu et al. developed a digital tissue phantom (DTP) platform for quantitative calibration and performance evaluation of spectral wound imaging devices and demonstrated the feasibility of such DTP platform by both in vitro and in vivo experiments.136 After the spectral calibration process, the wavelengths for the pixels along the spectral dimension of the data cube can be defined and the relationship between the true spectral position of the incoming light and the observed effect can be determined for further processing.

Besides the spectral calibration, a data preprocessing procedure is also needed to remove the effects of noises and artifacts and make the images suitable for quantitative analysis. During the spectral image acquisition, the noise counts accumulated on the detector may influence the pixel values beyond the true intensities. Various image artifacts can be generated by factors such as nonuniform illumination, dust on the lens surface, and pixel-to-pixel sensitivity variations of the detector.17 These effects in the original data can be corrected by some flat-field correction methods. In the past decades, different data correction methods have been proposed along with the development of the biomedical spectral imaging technology. Most of these methods utilize the radiance of standard white diffuse reference panels (for example, WS-1, Ocean Optics, FL) and the dark current data for the correction, which can be performed using the following equation:83,122,141,176,185Display Formula

Rsample(x,y;λ)=DNsample(x,y;λ)DNdark(x,y;λ)DNwhite(x,y;λ)DNdark(x,y;λ)·Rwhite(λ),
where Rsample(x,y;λ) and DNsample(x,y;λ) represent the relative reflectance image and the intensity image of the biological sample, respectively, DNwhite(x,y;λ) denotes the intensity values of bright calibration data, DNdark(x,y;λ) denotes the intensity values of dark current data, and Rwhite(λ) is the reflectance or factor of the white panel. More detailed derivations and discussions about this method can be found in the publication of Erives and colleagues.68 Although this equation is proposed for reflectance image correction, transmittance image data can also be corrected similarly. For example, a thin diffusing glass slide with 95% transmittance has been used to correct the transmittance images captured by spectral imaging microscopy.48,91,96 Then, the relative (or percent) reflectance instead of the absolute intensity data can be acquired for further data analysis.

Some commercial software has also been developed for biomedical spectral images analysis. For example, the CytoSpec program developed by Peter Lasch is a specialized software package with a large number of different functions, such as data import, spectral and spatial preprocessing, and uni- and multivariate image segmentation.186 Some other software associated with the commercial spectral imaging hardware also provide the basic calibration and preprocessing functions, such as the inForm™ intelligent image analysis software developed for the Nuance™ and Vectra™ multispectral imaging system, the HyperSpect™ cross-platform operating software for the 710VP Hyperspectral Imager, and the PARISS® software for the PARISS® system.

Spectral imaging systems are potentially useful for those applications in which subtle spectral differences exist in chemical constituents within a scene of spectral images. It has enabled applications in a wide variety of biomedical engineering studies ranging from individual biochemical species observed in in vitro samples to organs of living people. This section will concentrate on biomedical applications involving microcosmic and macroscopical, in vivo and in vitro. Some of the pioneering research works are mentioned here, but it is by no means a complete list.

In Vivo Spectral Imaging

In vivo optical imaging allows a noninvasive observation of living organisms and helps in understanding metabolic processes and disease-related changes in the body. As one of the optical diagnostic technologies, spectral imaging has been used for in situ tissues diagnosis in recent years. Among the openly accessible tissues, skin is one of the most suitable ones for in vivo spectral imaging. Afromowitz et al. developed a multispectral imaging system to evaluate burn depth of skin.112 They found that the spectral imaging method was more accurate in predicting burn healing than the traditional method. After that a spectral image-analysis system based on Fourier transformed spectroscopy combined with image processing has been used for the in vivo study of porphyrin localization in human skin lesions.107 These are some earliest studies on in vivo analysis on skin. Then Sowa et al. evaluated the hemodynamic changes in the early postburn period of skin by NIR reflectance spectroscopy and imaging.59,60,64 Cancio et al. further presented a study in which a spectral imaging system was used to assess the cutaneous manifestations of significant systemic events.67 Martinez proposed a noninvasive spectral reflectance method for mapping blood oxygen saturation in various types of wounds, ranging from burn injuries and diabetic-related ulcers, to skin grafts.187 Some other studies include determining skin ulcers,188 and spectral imaging of cancer skin fluorescence and reflectance.5,16,22In vivo confocal scanning laser microscopy has also been used to analyze the skin annexes, as well as cutaneous cells from different epidermal layers by both reflectance and fluorescence mode.189 More recent studies focused on 3-D HSI on small human skin areas,68 quantification of skin oxygen saturation and its relation to the traditional measures of wound healing,123 subcutaneous adipose tissue observation,176 ischemic wound assessment,136 and breast tumors.142 These studies show that noninvasive in vivo detection of skin surface using spectral imaging technology may be of value in the early assessment of burn injury and skin tumors.

Another important in vivo application of spectral imaging is retina detection. Some earlier studies focused on the development of the retinal spectral imaging system coupled with a standard ophthalmic fundus camera.144,190 Some researchers also discussed the analysis methods in detecting and characterizing the extent of retinal abnormalities according to the spectral images.41,191,192 A major disadvantage of the spectral imaging systems used in these studies is that they are fundamentally unsuitable for recording phenomena that are changing on a timescale that is shorter than the duration required in reconstructing the spectral data cube. Therefore, the snapshot spectral imaging technology has been proposed and used to capture retinal spectral images in recent studies. Harvey et al. developed an image replicating imaging spectrometer to capture spectral images of the retina and proposed a physical model that enables the oximetry map of retina.114,116 Johnson et al. and Ramella-Roman et al. obtained oxygen saturation maps of retina with a three-wavelength algorithm and found that the oxygen saturation were 95% for healthy subjects’ arteries and 30 to 35% less for the veins.18,122 Although the snapshot mode can capture the retinal spectral imaging quickly, the spatial and spectral resolutions are limited. In recent studies, LCTF-based retinal HSI systems have been developed to calculate the oxygen saturation of retinal vessel and detect retinal lesions.117,118,193 These studies show that a spectral imaging technique can be adapted to measure retina in vivo and map relative oxygen saturation in retinal structures. The confocal scanning laser ophthalmoscopy has also been used to evaluate retinal perfusion in the human eye.137,138 In addition, it can be combined with high-resolution spectral-domain optical coherence tomography (OCT) for simultaneous multimodal imaging of retinal pathologies, such as fluorescein and indocyanine green angiographies, IR and blue reflectance (red-free) images, fundus autofluorescence images, and OCT scans.194 Future work in this area will include clinical studies focused on spectrally characterizing retinal tissue, its diseases, and on the detection and tracking of the progression of retinal disease.118,191

The in vivo spectral imaging technology has also been used for tongue surface identification and oral cancer diagnosis. Li et al. introduced the spectral imaging technology to tongue diseases diagnosis for the first time.36 Yu et al. presented in vivo spectral images from a hyperkeratotic lesion on the ventral surface of the tongue to demonstrate clinical applicability.54 In recent studies, different kinds of HSI systems have been developed and used for tongue body segmentation, tongue color analysis and discrimination, tongue cracks extraction and classification, sublingual veins analysis, and malignant changes associated with oral cancer detection.4,3739,80,195 These preliminary experiments show that the spectral oral imaging system is superior to the traditional CCD-based methods because it can provide more information about the oral surface. However, further studies are needed on how to extract the quantitative features of the tongue surface and model the relationship between these features and certain diseases.

Besides the exterior organs and tissues, some intracorporeal organs also can be detected in vivo by the spectral imaging system coupled with colposcope, endoscope, and gastroscope. Parker et al. developed a noncontact in vivo hyperspectral diagnostic imaging device for the detection and localization of cervical intraepithelial neoplasia.135 Balas described an LCTF-based multispectral imaging system for the in vivo early detection, quantitative staging, and mapping of cervical cancer and precancer.77 Commercial systems such as the dynamic spectral imaging colposcope (DySIS, Dynamic Spectral Imaging System; Forth Photonics, Livingston, UK) have also been developed and used to detect cervical lesions.157,158,196 Similarly, Vo-Dinh et al. developed an AOTF-based endoscopic HSI system and evaluated it by a model of bronchial tubes (APM Inc.) and esophageal cancer tissues.5,16,22,84,85 Arnold et al. evaluated the hyperspectral video endoscopy demonstrator during several measurements in a clinical surrounding for thorax surgeries.87 In recent study, this technique has also been used in gastrointestinal diagnosis.88 Although most of these studies were mainly concentrated on the system design, and only preliminary results have been obtained, they have demonstrated the possibility of inspecting inner regions of the human body with additional diagnostic information by spectral imaging technology. There are also some other in vivo applications of spectral imaging systems, such as hemoglobin saturation evaluation in tumor microvasculature and tumor hypoxia development,91 characterization of activity-dependent changes in oxyhemoglobim, deoxyhemoglobim, and light scattering in brain,197 analysis of the oxygenation and blood flow through microvessel networks,198 normal and malignant muscle tissues detection,84 tooth decay detection,82,113,185 and flow cytometry.86 Guidance of surgical intervention, treatment, and real-time assessment of tissue response to therapy is another promising application of in vivo spectral imaging.16,199 Especially with the development of clinically relevant quantum dots (QDs) contrast agents suitable for spectral multiplexing, in vivo spectral imaging can help the surgeon to delineate tumor margins and to detect residual tumor cells or micrometastases. Singhal et al. summarized the advances in nanotechnology and optical instrumentation and how these advances can be integrated for applications in surgical oncology applications, such as tumor localization, tumor margins, important adjacent structures identification, mapping of sentinel lymph nodes, and detection of residual tumor cells.200 From the review, it can be found that the combination of nanoparticles and in vivo spectral imaging has particularly promising opportunities in surgical oncology, and it will be an important research trend in the future studies.

In addition, some researchers also use the diffuse spectral imaging system to perform the in vivo diagnosis. This technology has been used for early diagnosis and screening of malignant changes in the oral cavity. Some experimental results show the potential use of this method in routine clinical practice for instantaneous screening of oral cancer.201,