2.1.

Experimental Setup

As shown in Fig. 1, the integrated multimodal wound imaging system consists of a hyperspectral imaging module, a laser speckle imaging module, and a thermographic imaging module. The hyperspectral imaging module consists of a PIKA II hyperspectral line scan imager (Resonon, Bozeman, Montana) operated in a wavelength range from 400 to 900 nm, with a spectral resolution of 2 nm and a linear resolution of 640 pixels. The imager was driven by a motorized stage to scan along one direction at a designated step size and speed so that a two-dimensional hyperspectral image can be acquired within a specific field-of-view. A broad band halogen lighting system (Resonon) provided stable illumination during scanning. Before image acquisition, the system was calibrated following a standard procedure of setting exposure time, adjusting focal plane, compensating for dark current, and spectral response correction. For spectral response correction, a Fluorilon reflectance standard (Avian Technologies, LLC, Sunapee, New Hampshire) with the highest diffuse reflectance was used. To minimize the measurement bias caused by illumination condition variations, a National Institute of Standards and Technology (NIST) traceable diffuse reflectance standard was placed next to the region of interest (ROI) during hyperspectral imaging. Cutaneous tissue oxygenation maps were then reconstructed by spectral analysis of the hyperspectral data cube. A PeriCam PSI laser speckle system (Perimed Inc., Ardmore, Pennsylvania) was used to acquire the intensity fluctuations of the speckle pattern and converted them into blood perfusion characteristics using LASCA. A ThermalVision A40 infrared camera (FLIR Systems, Boston, Massachusetts) with a spatial resolution of 1.3 mrad and thermal sensitivity of 0.08°C was used to monitor thermal emissivity of the tissue surface relevant to inflammation, perfusion, and fat components. The above three imaging modalities were mounted on two orthogonally motorized linear stages and driven by a Labview program so that the field-of-view of each imaging modality can fit in the same ROI. The imaging system was mounted on a motorized table and its height is adjustable for appropriate field-of-views in biological tissues.

Fig. 1

Schematic diagram of the integrated multimodal imaging system. The system consists of a hyperspectral linear scan imager, a thermographic imager and a laser speckle imager, two motorized linear stages, a broad band light source, a National Institute of Standards and Technology (NIST) traceable white diffuser, and an operating computer.

2.2.

Occlusion Experiment

The multimodal imaging system was validated in an occlusion experiment where noncontact images of a healthy subject’s upper extremity were compared with concurrent measurements by standard contact sensors, as illustrated in Fig. 2(a). The standard sensors consisted of two PF5040 units and a PF5010 unit of a PeriFlux system 5000 (Perimed, Kings Park, New York). The PF5040 units (including two electrodes) measured transcutaneous tissue oxygen tension ($\mathrm{t}cp{\mathrm{O}}_2$). The PF5010 unit (including a 404 probe) was used for laser Doppler perfusion monitoring (LDPM). A pressure cuff was used to occlude the subject’s upper extremity. A PeriSoft software package was used to collect the measurement data. Before measurement, the subject was asked to relax in the test room for at least 30 min in order to stabilize the physiologic rhythms and reduce the temperature difference between the subject and the room. The subject’s upper arm was gently placed on the operating bed and kept steady. A probe 404 of the PF5010 unit and a $\mathrm{t}cp{\mathrm{O}}_2$ electrode of the PF5040 unit were attached side by side on the subject’s forearm skin for continuous monitoring of blood perfusion and oxygen partial pressure, respectively. The measurement of oxygen partial pressure was used to validate the multispectral oximetry technique because of an approximately linear correlation between oxygen saturation and oxygen tension in an oxygen saturation range from 20% to 80%.²² The focus points and the fields-of-view of the individual modalities in the multimodal imaging system were finely tuned so that the tissue ROIs defined within a square fiducial marker were imaged. Figure 2(b) shows the probes and the fiducial marker placed on the upper extremity of the subject. Before the experiment, a pressure cuff was placed around the upper arm of the subject and the baseline tissue parameters were monitored by the PeriFlux system 5000 and compared with that by the multimodal imaging system. After 2 min of baseline measurement, the upper arm was occluded by the pressure cuff at above $180\mathrm{mm}/\mathrm{Hg}$ for another 2 min. Then, the cuff was released for 2 min of reactive hyperemia. During the above process, dynamic changes of the tissue parameters were monitored by the multimodal imaging system and compared with the standard measurements.

Fig. 2

Occlusion experiment to validate the multimodal imaging system: (a) the standard sensors consisted of two PF5040 units and a PF5010 unit of a PeriFlux system 5000; (b) the probes and the fiducial marker on the upper extremity of a healthy subject in the experiment.

2.3.

Wound Healing Monitoring

Tissue parameter changes during the wound healing process were continuously monitored by the multimodal imaging system in a clinical trial following an IRB approved test protocol. A healthy subject rested in a test bed for 30 min with a fiducial marker of $40\mathrm{cm}\times 40\mathrm{cm}$ square frame placed on the lower extremity. A cutaneous wound of 3 mm in diameter was introduced by a biopsy punch at the center of the fiducial marker. Multimodal images were acquired before and after biopsy using hyperspectral, laser speckle, and thermographic imaging modalities. The healing of the wound was monitored by successive imaging sessions on the designated days for 1 month after biopsy. In each imaging session, the fiducial marker was placed on the wounded skin with its center aligned to the center of the wound. The position and the orientation of the fiducial marker remained consistent for different imaging modalities and during individual image sessions.

2.4.

Imaging Algorithms

Imaging algorithms in the integrated multimodal imaging system were developed to accomplish the following two primary tasks: (1) quantitative assessment of tissue oxygen and tissue perfusion, and (2) coregistration of the images acquired by different modalities and in different imaging sessions. Many conventional hyperspectral imaging systems provide only relative assessment of tissue oxygenation because of the measurement artifacts induced by skin pigmentation, scattering, and background absorption. We have previously proposed a multispectral imaging algorithm that uses the wide gap second-derivative reflectance spectra for quantitative assessment of tissue oxygenation.²³ This algorithm simulated the reflectance spectra of multilayered skin tissue at different oxygen saturation, blood, tissue scattering, and melanin levels using an empirical model calibrated by Monte Carlo simulation. Based on the simulation results, a second derivative ratio parameter was derived and optimized for multispectral reconstruction of tissue oxygenation regardless of variations in tissue conditions. Previous experiments have demonstrated that the proposed multispectral imaging algorithm was able to monitor cutaneous tissue oxygenation with minimal artifacts at different tissue conditions.²⁴ In this paper, a modified hyperspectral imaging algorithm was adopted for in vivo imaging of wound tissue oxygenation based on the wavelength range and the high resolution of the hyperspectral imager. The algorithm treated the apparent absorption of skin tissue from 520 to 585 nm as a linear combination of oxy-hemoglobin, deoxy-hemoglobin, and a linear function modulated by background scattering and melanin concentration.²² Diffuse reflectance $R$ at the designated wavelengths was calculated by taking the ratio between the acquired image intensity and the intensity of a National Institute of Standard and Technology (NIST) traceable reflectance standard. Hyperspectral tissue absorption spectrum was calculated by $A=-\log 10(R)$. The artifacts of scattering and melanin were corrected by assuming a linear contribution ($a+b*\lambda$) to the absorption spectrum. Therefore, $A=a+b*\lambda +C*\epsilon (\lambda )$, where $a+b*\lambda$ is the contribution from melanin and scattering, $C$ is the total hemoglobin concentration and $\epsilon (\lambda )$ is extinction coefficient of four isosbestic points where $\lambda$ are 530, 545, 570, and 584 nm. $a$ and $b$ can be obtained by a linear fitting method. Then the corrected absorption spectrum $A\prime (\mathit{\lambda })=A-a-b*\mathit{\lambda }$ is the absorption only from hemoglobin. Thus, $A\prime (\mathit{\lambda })=C_1*{\mathit{\epsilon }}_1(\mathit{\lambda }){\text{\hspace{0.17em}+\hspace{0.17em}}C}_2*{\mathit{\epsilon }}_2(\mathit{\lambda })$, where $C_1,C_2,{\mathit{\epsilon }}_1$ and ${\mathit{\epsilon }}_2$ are the concentrations of oxy- and deoxy-hemoglobin, extinction coefficient of oxy- and deoxy-hemoglobin, respectively. In order to obtain $C_1$ and $C_2$, the linear fitting method Levenberg–Marquardt algorithm was used to minimize the mean square error. Therefore, the oxygen saturation is ${\mathrm{StO}}_2=C_1/(C_1+C_2)$. For an ROI around the wound tissue, we averaged oxygen saturation of all points in the ROI and took the average value as the oxygen saturation of the ROI.

Perfusion characteristics of wound tissue were acquired with an LASCA method. Illuminating biologic tissue by coherent and monochromatic laser light produces an interference pattern (i.e., a speckle pattern) on the tissue surface. The intensity fluctuations of the speckle pattern were recorded by the laser speckle imaging module. By analyzing these intensity fluctuations, the perfusion characteristics of the tissue could be obtained. In LASCA, spatial contrast was defined as the standard deviation of the intensity fluctuations divided by their mean intensity, in a small region of the image. To calculate the contrast from intensity and variance pairs, the following equation should be used: $C=\beta (\sqrt{V}/I)$, where $C$ is the spatial contrast, $V$ and $I$ denote the variance and the intensity, respectively, and $\beta$ is the coherence factor that is determined by a calibrate experiment. The perfusion was calculated from the contrast as follows: $P=k[(1/C)-1]$, where $P$ is the perfusion of the small region of the image and $k$ is the signal gain factor. The signal gain factor was calibrated to ensure the designated perfusion values on a flux standard just as the coherence factor was.

Figure 3(a) shows the hyperspectral, thermographic, and laser speckle images of the same wound tissue on different days before and after skin biopsy. These images were acquired by different modalities with different focus lengths, orientations, and fields-of-view. Therefore, priori coregistration of the selected ROIs was necessary. Considering that the fiducial marker remained stationary on the subject’s lower extremity throughout each imaging session, it was rational to use the geometric features on the fiducial marker for coregistration of the multimodal images in each session. In this regard, four points at the inner corners of the fiducial marker were selected as the control points. As shown in Fig. 3(a), the control points were determined by calculating the intersection of two perpendicular lines that represented the inner edges of the fiducial markers. Once the corresponding control points in the multimodal images acquired in the same image session were identified and paired, coregistration of these images was achieved by applying a transform matrix that translated and rescaled each image for identical matching of the control points.

Fig. 3

Image coregistration in wound healing monitoring, (a) images from different imaging sessions and different imaging modalities; (b) selecting corresponding pores as control point pairs; (c) fusion result of the images from the hyperspectral imager at different imaging sessions; (d) fusion result of the images from the three modalities at the same imaging session.

The fiducial marker method was not appropriate for coregistering multimodal images acquired on different days during the wound healing procedure because the position and the orientation of the fiducial marker cannot be kept consistent in different imaging sessions. To overcome this problem, we used pores in the test subject’s skin as a natural marker for coregistration. These biologic features were intrinsic and did not change dramatically during the wound healing process. To implement the coregistration algorithm, we first defined an ROI within the hyperspectral images acquired in different imaging sessions. Then, we identified three or more characteristic pores within the ROI, found their counterparts in the individual hyperspectral images, and calculated their weighted centers. The accurate positions of these pores in different imaging sessions were determined and the coordinates of the corresponding control points were calculated, as exemplified in Fig. 3(b).

Once three or more pairs of control points were determined between two un-registered images, a spatial transformation matrix could be derived by an affine transformation method. In affine transformation, the planar feature of the multimodal images might be translated, scaled, or sheared independently: $[\begin{array}{cc}u& v\end{array}]=[\begin{array}{ccc}x& y& 1\end{array}]*T$, where $[\begin{array}{cc}u& v\end{array}]$ and $[\begin{array}{cc}x& y\end{array}]$ denote the coordinates of the control points in two images, respectively; $T$ is a $3\times 2$ matrix that contains six unknown elements, thus at least three control point pairs were needed to solve $T$. If more control point pairs were selected, a least-square solution was utilized to enhance the accuracy. After the spatial transform was obtained, the image can be transformed according to $T$ and can be registered to a single fusion image. Figures 3(c) and 3(d) display the fusion result of the images from different modalities and different imaging sessions, respectively.

Image coregistration based on the intrinsic biologic features such as pores is a new method that provides tissue information within a consistent ROI on different days of the wound healing procedure. In order to evaluate the accuracy of this coregistration method, we placed four round objects on the subject’s skin within the ROI to simulate different skin geometric features. Hyperspectral images were acquired from different points of view and different fields-of-view, simulating the scenario of different imaging sessions, as shown in Figs. 4(a)–4(c). Figures 4(d) and 4(e) show the corresponding images after pore-based coregistration. According to the figures, appropriate coregistration is achieved for not only the pores but also the round objects placed within the ROI. Quantitative analysis of the coregistrated round objects acquired in different imaging sessions yields an overall overlapping rate of above 95%, indicating the technical feasibility for tracing the same ROI with accuracy in different imaging sessions of the wound healing process.

Fig. 4

Coregistration results based on pores. (a–c) three hyperspectral images from various sessions, three circles (blue, green, and red) on the images denote the pores selected for coregistration. Numbers 1 to 4 denote the four round objects utilized to check the coregistration accuracy, respectively; (d) image fusion between (a) and (b) after pore-based coregistration; (e) image fusion between (a) and (c) after pore-based coregistration; (f) overlapping rates of the objects in different imaging sessions after pore-based coregistration. The averaged overlapping rate between images (a) and (b) is 97.3%, while that of (a) and (c) is 98.9%.

3.1.

Occlusion Dynamics

With the above experimental setup and algorithm development, the dynamics of the functional characteristics of the tissue were acquired and analyzed. Figure 5 shows the dynamic changes of tissue functional parameters during a PORH process that involves successive stages of baseline, occlusion, and recovery. Figure 5 plots, from top down, the dynamic changes of the transcutaneous tissue oxygen partial pressure ($\mathrm{t}cp{\mathrm{O}}_2$, monitored by PF5040), tissue perfusion (measured by LDPM), tissue ${\mathrm{StO}}_2$ (reconstructed from the acquired hyperspectral images), tissue perfusion (reconstructed from the acquired laser speckle images), and thermographic measurement of heat emission. According to the figure, all the above tissue parameters demonstrate the corresponding dynamics in response to three stages of the PORH process. However, their response rates are different, as shown in Fig. 5. Unlike the perfusion and the oxygen saturation profiles that show agile responses to vessel occlusion and recovery, the dynamics of $\mathrm{t}cp{\mathrm{O}}_2$ and thermal emission are relatively lagged and slow. The retarded $\mathrm{t}cp{\mathrm{O}}_2$ may be explained by slow diffusion and transport of oxygen from capillary blood to red blood cells in response to tissue metabolic changes.^25,26 In addition, the $\mathrm{t}cp{\mathrm{O}}_2$ electrode placed on the surface of the skin tissue also contributes to the delayed response. The retarded thermographic response is associated with the convective heat transfer by the capillary blood flow, the conduction of heat from capillary blood to skin tissue, and the heat dissipation to the surrounding environment.²⁷ At the normal physiologic condition, a relatively constant skin temperature is maintained by keeping a balance between heat transfer from capillary blood to surrounding tissue and heat dissipation from skin tissue to environment. In the case of vascular occlusion where blood perfusion is inhibited, the rate of heat transfer from vascular blood to skin tissue is significantly reduced, whereas heat dissipation from skin tissue to environment remains similar, leading to a decreased skin temperature profile as plotted in Fig. 5(e). Considering the time for heat transfer from capillary blood to skin surface and the thermal inertia of the skin tissue, the thermographic measurements are relatively slow in response to vascular occlusion and recovery. The above experiment results have verified the capability of our multimodal imaging system for simultaneous monitoring of multiple tissue parameters in a noninvasive and quantitative fashion. Further analysis of these tissue parameters and their responses to a dynamic process such as PORH may help us to establish a mathematical model of tissue perfusion and metabolism for quantitative identification of tissue anomalies.

Fig. 5

Occlusion dynamics during the postocclusive reactive hyperemia process. Blue lines denote $\mathrm{t}cp{\mathrm{O}}_2$ and tissue perfusion measured by the standard contact sensors, respectively, (a and b); while red lines denote ${\mathrm{StO}}_2$, perfusion, and heat emission of the tissue obtained from the integrated multimodal imaging system, respectively (c–e).

3.2.

Wound Healing Process

After the multimodal images were acquired and coregistered throughout the whole wound healing process, the dynamic changes of tissue functional parameters within the specific ROIs can be reconstructed and analyzed. For this purpose, eight ROIs were selected on the wound tissue and the tissue parameters were averaged within each ROI. ROI #1 represented the area that exactly covered the introduced wound, whereas ROIs #2 to #8 denoted the adjacent torus areas from the wound center to surrounding tissue, respectively. Figures 6(a), 6(b), and 6(c) display dynamic changes of oxygen saturation, perfusion and heat emission of wound tissue and surrounding tissue during the healing process according to ROIs, respectively. It is observed from the baseline hyperspectral images that the oxygen saturation levels are even before the introduction of a cutaneous wound. As the wound is introduced through biopsy, oxygen saturation levels decrease gradually from the center of the wound to the outside areas regardless of the date of the healing process. The only exception is that the oxygen saturation of ROI #1 (i.e., the center of the wound) is lower. The reduced oxygenation at the center of the wound is due to coagulation and scar formation within 2 weeks after biopsy. The oxygen saturation of ROI #1 increases in the last two imaging sessions of the healing process as the scar drops off and the new skin tissue is exposed. For the assessment of the whole wound tissue, oxygen saturation increases rapidly immediately after biopsy, followed by a slow decrease until it reaches the lowest point on Day 7. After Day 7, the oxygen saturation levels in the most of the ROIs increase until they reach the levels similar to that of the baseline before biopsy.

Fig. 6

Results of wound healing monitoring. (a), (b), and (c) represent the dynamics of ${\mathrm{StO}}_2$, perfusion, and heat emission of regions of interest (ROIs) around wound tissue in the wound healing process, respectively.

Similar to oxygen saturation, the perfusion in different ROIs is evenly distributed before the introduction of the cutaneous wound. After biopsy, the perfusion dynamics of different ROIs vary significantly. However, the averaged perfusion level over all the ROIs increases immediately after biopsy for 3 days, fluctuates for 2 weeks, and sharply decreases on Day 16. Such a dynamic change of the vascular perfusion may be associated with the tissue proliferation and vascular remodeling of the wound healing process. The perfusion levels within various ROIs finally converge to that of the baseline before biopsy. Similar to the case of oxygen saturation, the perfusion levels of the ROIs around the wound decrease from the center of the wound to the surrounding tissue, indicating the increased delivery of oxygen and nutrition to facilitate wound healing.

The thermographic images are also analyzed to reveal the heat emission changes during the wound healing process. To overcome the measurement deviation due to the room temperature fluctuation in different imaging sessions, we set the thermo graphic measurement in ROI #8 as a reference point to calculate the relative thermal emissions of other ROIs, as plotted in Fig. 6(c). It can be seen from the figure that the relative heat emission of the tissue decreases from the center of the wound to the edge immediately after biopsy till Day 9. However, this distribution is reversed after Day 14. This phenomenon implies that inflammation may spread from the center of the wound to the boundary.

With dynamic imaging of tissue oxygenation, perfusion, and thermal emission, it is possible to identify different phases of the wound healing process and provide quantitative guidance for different therapeutic options. However, the functional dynamics of these tissue parameters do not follow the coincident pattern, making it important to adopt the proposed multimodal imaging system for appropriate detection of chronic wounds and optimal design of the compatible therapy.