Research Papers: Imaging

Hybrid optical imaging technology for long-term remote monitoring of skin perfusion and temperature behavior

[+] Author Affiliations
Nikolai Blanik

RWTH Aachen University, Helmholtz-Institute for Biomedical Engineering, Chair for Medical Information Technology, 52074 Aachen, Germany

Abbas K. Abbas

RWTH Aachen University, Helmholtz-Institute for Biomedical Engineering, Chair for Medical Information Technology, 52074 Aachen, Germany

Boudewijn Venema

RWTH Aachen University, Helmholtz-Institute for Biomedical Engineering, Chair for Medical Information Technology, 52074 Aachen, Germany

Vladimir Blazek

RWTH Aachen University, Helmholtz-Institute for Biomedical Engineering, Chair for Medical Information Technology, 52074 Aachen, Germany

Steffen Leonhardt

RWTH Aachen University, Helmholtz-Institute for Biomedical Engineering, Chair for Medical Information Technology, 52074 Aachen, Germany

J. Biomed. Opt. 19(1), 016012 (Jan 17, 2014). doi:10.1117/1.JBO.19.1.016012
History: Received June 25, 2013; Revised November 25, 2013; Accepted December 3, 2013
Text Size: A A A

Open Access Open Access

Abstract.  Photoplethysmography imaging (PPGI) and infrared thermography imaging (IRTI) are contactless camera-based measurement methods for monitoring a wide range of basic vital parameters. In particular, PPGI enhances the classical contact-based photoplethysmography. Approved evaluation algorithms of the well-established PPG method can easily be adapted for detection of heart rate, heart rate variability, respiration rate (RR), respiratory variability (RV), and vasomotional activity with PPGI. The IRTI method primarily records temperature distribution of the observed object, but information on RR and RV can also be derived from IRTI by analyzing the development of temperature distribution in the nasal region. The main advantages of both monitoring methods are unobtrusive data acquisition and the possibility of assessing spatial assignment between vital parameters and body region. Hence, these methods enable long-term monitoring or the monitoring of effects with special local characteristics. Because the two systems supplement each, a combined hybrid application is proposed and its feasibility discussed.

Figures in this Article

The monitoring of basic vital signs, like heart rate (HR) or respiration rate (RR), is a fundamental task in clinical care. For this purpose, various techniques have been established and are available in ICU monitors, anesthesia devices, etc. For continuous long-term monitoring, or for frequently repeated measurements, an unobtrusive way of acquiring data becomes increasingly important. However, even when unobtrusive, most methods currently available still require direct contact between a sensor element and a specific body region. Although in many cases direct skin or tissue contact is feasible, for some fields of application, skin contact is either undesirable or is not possible. For example, in patients with extensive burns or lesions that involve large areas, adhesive or mechanically attached sensors (like ECG electrodes or finger clip pulse oximeters) cannot be applied in or on the wounded areas. A similar situation arises with the use of sensors in neonatal intensive care. Because of the immature skin (particularly in preterm infants), direct skin contact with sensors may cause irritation or skin injury, thereby increasing the risk of infection.1,2

To prevent the possible harmful side-effects of contact monitoring methods, a noncontact, camera-based alternative for continuous monitoring is proposed. This involves two camera systems: (1) photoplethysmography imaging (PPGI) for active optical measurement of skin perfusion and (2) passive infrared thermography imaging (IRTI) for measurement of surface temperature. In addition, the spatial resolution of the cameras allows one to assess the local distribution of vital parameters. With these compact and user-friendly cameras, the measurement setup can easily be adapted to a wide range of applications.

Therefore, the basic principles of the two camera-based systems are described and the results of preliminary experiments are presented to illustrate the possibilities with these techniques. Finally, a combination of contactless measurements based on space-resolved IRTI and simultaneous PPGI is proposed for a novel hybrid camera-based monitoring system.

The application of camera-based monitoring systems offers an ideal opportunity to realize contactless recognition of vital signs, as well as a spatially resolved functional assessment and interpretation of the obtained data. The two optical systems are described below.

Photoplethysmography Imaging

The PPGI technique is an advancement of the classical and well-established photoplethysmography (PPG).35 Nowadays, PPG is widely used in many medical disciplines and is appreciated by physicians because of its simple design and relatively low cost per examination. PPG systems with optical sensor technology in the near-infrared (900nm) light range can noninvasively detect changes in blood volume (related to arterial and venous hemodynamics) in the dermal blood vessel network by registering the optical attenuation A of transilluminated tissue, according to the Beer-Lambert law.6Display Formula

A=iεicil=log(II0),(1)
where ε is the extinction coefficient of the absorbing tissue species i, c is its concentration, l is the path length through this tissue, and I0 and I are the intensities of the incident and transmitted light, respectively.

In this active optical sensing system, one or more selective light sources and one or more light detectors are integrated into the optical sensor unit, which can work in reflection or transmission mode.

Vital signs that can be observed include HR and RR (and their variability), which may provide information on the autonomic nervous system, perfusion phenomena with frequencies below respiration, venous reflux, venous refilling rates, and other diagnostically relevant physiological indexes or signals.710 Reflection mode sensor technology (rPPG) can be applied to practically all body areas, thus overcoming the limitation of transmission (tPPG) sensors, which can only be applied to thinner body parts, such as the fingertip or ear lobe.

In rPPG, distances between light source and light detector are in the order of a few millimeters [Fig. 1(b)], and typical penetration depths of light into the tissue are around a few millimeters, depending on the wavelength and sensor geometry.7

Graphic Jump LocationF1 :

History of reflective photoplethysmography, from (a) Hertzman’s first optoelectronic photoplethysmograph3 to (c) modern remote photoplethysmography imaging (PPGI).16

Using two or more wavelengths at the same time also allows one to measure arterial oxygen saturation in skin, for example, using the pulse oximetry method, see, e.g., 11.

In PPGI, the classical reflective-mode, skin-contact PPG sensor is replaced by a highly sensitive camera detector and an external illumination unit consisting of multiple light-emitting diodes.1216 In particular, to detect the weak light modulation caused by blood hemodynamics, our setup17 uses the Pike210B camera (Allied Vision Technologies Ltd., Hod-Hasharon, Israel) chosen due to its high dynamic range of 96 dB and its high readout speed of up to 31 full frames per second. The imaging sensor is a Kodak KAI-2093 type 1 progressive scan CCD (diag. 15.3 mm) with a pixel resolution of 1928×1084 and a spectral range of 320 to 950 nm with a quantum efficiency of >40% at 480 nm.

The setup shown in Fig. 2 allows one to measure arbitrary parts of the skin surface and to simultaneously assess dermal perfusion in different skin regions. The PPGI system can assess local changes in perfusion patterns and create a functional mapping of the perfusion status. After recording a video sequence of the skin surface, the operator can select different regions of interest (ROIs; virtual sensors) for which the backscattered light intensity is calculated. The resulting signals contain the same information as the classical PPG signals. Of considerable importance is the possibility to obtain measurements even in wounds, transplanted skin, or on a neonate’s sensitive immature skin.

Graphic Jump LocationF2 :

(a) Early PPGI setup realized at RWTH Aachen University in 2000.12 (b) Screen shot of a typical perfusion signal.

The PPGI camera is connected to a high-performance PC via FireWire (IEEE1394b). The PPGI software was developed to control the camera settings, capture the image sequence, and perform postprocessing and analysis. Currently, the PPGI software is capable of acquiring the video in online mode and saving it for offline processing. In online mode, the live video can be displayed dynamically. The maximal sampling rate amounts to 31 fps at full resolution, but it can be increased by reducing sensitivity, acquiring only a subarray, or by binning. The file size of a typical PPGI sequence with spatial resolution of 640×480pixels, 15 Hz sampling frequency, and 1-min duration is 500MB. At full frame and maximal readout speed, 1-min recordings will produce >6GB of data. Hence, to prevent extensive storage consumption, the recording parameters should be limited according to the analyzing requirements: the necessary image detail and sampling rate. Inside the PPGI frames, different sizes and positions of ROIs can be selected so that virtual PPG sensors of variable measuring window can be freely selected at different places. Here, typical sizes of these ROIs range from 10 to 50 pixels edge length. By calculating the average pixel values of the ROIs, the reflected optical signal (the PPG signal), which corresponds to blood volume changes, can be monitored. This signal describes the skin hemodynamics inside the ROI in the time domain.

For offline processing, first, the video is calibrated. Calibration includes equally intervalled resampling and motion compensation. Motion artifacts may occur due to movements of the patient/infant, or the patient’s body part in relation to the camera. Calculating differences relevant to subsequent images provides a first indication of the occurrence of movement. At points in time with motion artifacts, the calculated differences increase considerably, especially in regions with high contrast. Tracking of easily detectable edges in the images allows conclusions to be drawn about, e.g., an infant’s activity or, when monitoring the rise and fall of the chest contour, the registration of breathing activity.

For evaluation of perfusion information from PPGI sequences, motion artifacts are generally disturbing. On one hand, a clear spatial assignment of detected signals to certain body regions is no longer possible. On the other hand, the detected signal itself is corrupted by coupled frequency components of the movement artifact. Depending on the movements, the detected signal may be totally disrupted. To avoid this, depending on the type of expected movement (e.g., slight, strong, translational, or rotatory), the computational time available, and required accuracy of the result, various compensatory techniques can be applied.15 In our research, we mainly use block-matching algorithms, hierarchical motion approximation with the modified Newton algorithm, or calculation of motion constraints. The temporal effort for these methods depends on the image resolution, the size and number of tracked ROIs as well as the required accuracy. On modern PCs, the movement compensation of a single ROI with our experimental setup can be computed in quasi-real-time. For IRTI, these motion detection and compensation methods are applied in an analogous way.

Infrared Thermography Imaging

Similar to PPGI, IRTI is a contactless but passive imaging technique for spatial and temporal registration of an object’s surface temperature.

To understand the basic principle of IRTI, it is essential that any physical object with a temperature above the absolute zero point (0 K) emits radiation. Physically, the specific spectral radiance M(λ,T) of ideal black bodies is described by Planck’s law.18Display Formula

M(λ,T)=2πhc2λ5·(ehcλkT1)1.(2)

In this equation, T is the black body’s temperature, h is the Planck constant, c is the speed of light in a vacuum, λ is the wavelength, and k is the Boltzmann constant. For selected temperatures, the Planck spectra are shown in Fig. 3.

Graphic Jump LocationF3 :

Specific spectral radiance for black bodies at different temperatures. (a) Image shows distribution at 3000 to 6000 K. (b) Image for human body surface temperatures. For temperatures around 6000 K, the maximum intensity is radiated in the visible spectrum. For lower temperatures, this maximum decreases and shifts to the long-infrared region. The maximum spectral sensitivity of the infrared thermography camera (VarioCam hr head) lies 9.5μm in the far-infrared region19 and is therefore suitable for the study of human body temperature.

The maximal emitted intensity for a fixed-size wavelength interval is strongly dependent on the body’s temperature. In fact, according to Stefan-Boltzmann’s law, the complete emitted radiation power P in the whole spectrum is a function proportional to the body’s temperature to the power of four.19Display Formula

P(T)=σ·A·T4,(3)
with σ=5.67·108Wm2K4, the Stefan-Boltzmann constant and A the object’s surface.

A shift of the maximum to longer wavelengths for lower temperatures can also be observed. This effect is described by Wien’s displacement law.4Display Formula

λmax=bT,(4)
with b2897.8(μm·K), the Wien’s displacement constant.

For typical skin temperatures (ca. 35°C), this maximum shifts to 9.5μm lying in the infrared C-band (IR-C). To draw conclusions about an object’s temperature by measuring its passive radiant emittance, the sensing device should be most sensitive in this wavelength region.

To record thermal image sequences, the VarioCam hr head (InfraTec GmbH, Dresden, Germany) is used. It possesses a geometric resolution of 384×289pixels and a sampling frequency up to 50 Hz.20 It is sensitive in the long-wave infrared band (7 to 14 μm) and most sensitive at 9.5 μm. Its dynamic range of 72 dB allows measurement of temperatures from 40 to +1200°C with best temperature resolution at 30°C (better than 0.03 K). Recorded data may be transferred via FireWire to a PC. The file size of a typical thermal video sequence of 1-min duration at 50 Hz is 400MB.

The recorded signal intensity correlates with the incident radiation weighted with the sensor’s sensitivity characteristic. With respect to internal calibration of the camera with regard to damping of the object lens, sensor, and camera temperature, and duration of exposure, this intensity can be correlated to the black body temperature. The distance of the observed object relative to the camera does not affect the detected temperature because the radiation density of a fixed-sized object decreases with the square of the distance, but at the same time, the size of an object observable with a single sensor pixel increases with the square of the distance.

However, to obtain precise measurements an additional calibration is required because in this way only temperatures of ideal black bodies can be measured exactly. However, objects of interest generally act like gray bodies or selective radiators. Their specific radiation Mobj is less than the specific radiation of an equivalent black body MBB. Display Formula

Mobj=ε(λ)MBBε[0,1].(5)

The emissivity coefficient ε is dependent on the wavelength of the object’s material and its thickness. In the wavelength interval of the sensor’s sensitivity range, ε can be defined unambiguously. Typical values of ε for different materials are provided in reference tables or by the camera manufacturer.20,21Table 1 presents emissivity constants for a selection of different materials valid in the IR-C band (8 to 14 μm).

Table Grahic Jump Location
Table 1Emissivity coefficients ε in the infrared C band (8 to 14 μm).20,21

Although the aberration of skin from an ideal black body is relatively small (0.97 versus 1), its influence is not negligible for accurate measurement. According to Eq. (5), an aberration of 0.03 of the detected radiance intensity leads to a nontolerable error of 2 to 3 K for measured skin temperature. However, knowing the emissivity coefficient, the equivalent black body radiation and thus the temperature can be estimated.

An emissivity coefficient close to 1 also indicates almost black body behavior. This means that nearly all incident radiation is absorbed. Hence, the penetration depth of electromagnetic radiation into the subject is close to 0. In reverse, all measured radiation from a black body has its origin at its surface. For example, radiation from deeper skin layers is already absorbed by the overlying layers. As a consequence, the measured temperature corresponds to the upper surface skin temperature.

Besides calibration according to the type of material involved, the measuring section itself can influence the recorded temperatures and therefore needs additional calibration and compensation steps. Figure 4 shows the potential sources of errors depending on the measurement geometry.22

Graphic Jump LocationF4 :

Composition of detected radiation in infrared thermography (IRT) monitoring (modified from 21). The amount of radiation entering the measuring section (1) toward the IRT camera (2) is a sum of the emitted radiation (3) of the measured object (4) according to its temperature, environmental radiation (5) reflected on the object’s surface (6) and background radiation (7) transmitted through the object (8). Therefore, the total detected radiation (9) depends on the absorption and emitting characteristics of the media (10) in the measuring section.

On one hand, total radiation coming from the measured surface is generally a sum of three separate components. The first is the emitted radiation of the body (which we are interested in), the second is the transmitted radiation from the background through the body, and the third component is the radiation reflected on the object’s surface from the environment. On the other hand, total radiation detected by the camera differs from the sum of the radiation coming from the measured surface depending on the media in the path of the rays. Here, part of the radiation might be absorbed, or even new components might be emitted toward the camera, according to the media’s material property, density, and temperature.

For medical IRT applications, the influence of transmitted radiation through tissue can be neglected because the human body is not transparent in the IR-C band (Table 1). Reflections at the surface are strongly suppressed because of the nearly ideal black body behavior of skin, absorbing incoming radiation; however, minor parts are reflected toward the camera. The percentage of this reflected radiation of the total radiation is a function of the environmental radiation intensity. For example, monitoring of neonates under an IR heating unit requires compensation of the reflected components. Disturbing media in the pathway are not totally transparent objects, like windows or even air moisture.23 According to Eq. (1), their effect on measurement increases when increasing the measuring distance or thickness of the material.

As is usual, the environmental and geometric conditions of IRTI measurements are assumed to be constant, or at least changing only slowly (e.g., room temperature or monitoring of a calm patient). Therefore, even disturbing influences will not change during measurement. This allows one to either calculate the offset in the temperature measurement if all environmental factors are known or compensate for their influence by additional calibration measurement with known references (e.g., measurement on dummies with known temperature and emissivity).

The compact and nonintrusive nature of the camera-based monitoring techniques allow multiple and flexible applications. The two systems have been tested and their feasibility confirmed in preliminary studies with volunteers or patients.

Results from PPGI

For the PPGI system, the extraction of vital parameters is comparable to the classical PPG. Due to the spatial resolution, multiple time series (instead of a single one) need to be analyzed. Time series can be obtained either from the mean time-varying gray values inside the ROIs or for every pixel (or pixel array) separately. Algorithms used for extraction of vital signs in classical PPG are applicable one-to-one on these time series. Therefore, vital parameters (like HR or HR variability) can also be recovered from the PPGI sequence. Depending on the observed perfusion frequency band, in addition to rather, other rhythmic phenomenon (like breathing pattern or vasomotor activity) can also be monitored.

Figure 5 shows a typical PPGI recording with the corresponding time series of three ROIs. The sequence was recorded on the left hand with a small wound on the middle finger [Fig. 5(a); red ROI). For each ROI, the time signal can be calculated by estimating the mean gray value inside the ROI for each frame successively. Hence, the absolute perfusion amplitudes may feature strong inter- and intrapersonal variability; the signals were normalized by dividing them through their dc components to improve comparability. In all three time series, a signal component that strongly correlated with the heartbeat is clearly visible at 74bpm [Fig. 5(b)]. However, the perfusion patterns from healthy skin and from the wound on the middle finger show significant differences. When examining only the heartbeat, its amplitude is slightly increased inside the wound, whereas the slower rhythms at 0.1Hz are markedly reduced. More detailed analysis of signal components can be obtained by applying band-pass filters to the signals. Depending on the selected cut-off frequencies, for example, either rhythms correlated to the respiration frequency band [Fig. 5(c)] or vasomotor rhythms [Fig. 5(d)] around 0.1 Hz can be extracted. Please note that minor inaccuracies in the first and last few seconds of the band-pass filtered sequence are due to filter characteristics. In this example, although the amplitude of the respiration correlated signal is much weaker than the heartbeat or 0.1-Hz rhythm, phase synchronous patterns can be monitored on any region on the skin. In contrast, 0.1-Hz (vasomotor) rhythms differentiate between wounded and healthy skin. Inside the wound, the slow rhythms have almost vanished. In healthy skin, even in adjacent ROIs, local variations are clearly visible.

Graphic Jump LocationF5 :

Example of PPGI analysis. (a) PPGI sequence of subject’s left hand with a small wound on the middle finger. Inside the first frame, three regions of interest (ROIs) are indicated: one inside the wound (red) and two close to each other on the back of the hand (blue and green). (b) Corresponding time signals for each ROI. (c) Band-pass filtered signals with cutoff frequencies at 0.2 and 0.6 Hz correlated with respiratory activity. (d) Band-pass filtered signals with cutoff frequencies at 0.02 and 0.2 Hz representing slow signal rhythms. (e) Fast Fourier transformation of the red signal. Five peaks in the frequency domain are visible: 0.1Hz rhythm (1), respiratory rhythm (2), heart rate (3), and harmonics of heart rate (4 and 5).

A similar behavior can be monitored with PPGI recordings on the forehead of a relaxed, seated subject. Here, spatial rhythm fluctuations in dermal perfusion are visible. This phenomenon is best observed in actual video sequences of the PPGI recordings as blood volume clouds, which move across the forehead in an organized but complicated pattern.

Figure 6 aims to illustrate this with screenshots of the recording; the actual video is available as an electronic supplement on the website of the Journal of Biomedical Optics.

Graphic Jump LocationF6 :

Phenomenon of distributed blood volume clouds observed with PPGI on the forehead of a seated subject in a relaxed steady state. The postprocessed and colored images show perfusion-coded images taken at 14, 16, 50, and 69 s of a PPGI video sequence (Video 1, MPEG, 2 MB) [URL: http://dx.doi.org/10.1117/1.JBO.19.1.016012.1].

An alternative approach to analyzing the frequency components of recorded time series (superposition of all rhythmic phenomena, of venous and arterial origin, in the dermal perfusion) is visualization of these phenomena by analyzing beat-to-beat distances of subsequent heartbeats (Fig. 7). It is assumed that the rhythmic changes of HR correlate with the respiratory sinus arrhythmia. For this kind of arrhythmia, the HR increases during inspiration and decreases during expiration. A combination of these video signal postprocessing strategies may provide new insight into perfusion dynamics.

Graphic Jump LocationF7 :

Beat-to-beat analysis of PPG signal. (a) Raw PPG signal with points of identical phase indicated (here: maxima) of single heart beats. (b) Beat-to-beat distance of the subsequent heart beats.

Results from IRTI

Following approval from the Medical Ethics Committee of RWTH Aachen University Hospital (EK032/09 August 19, 2009), the development of temperature distribution in seven neonates inside an incubator was monitored for 10 min. The IRT setup [Fig. 8(a)] was calibrated taking into account environmental humidity, radiant heat sources of the incubator unit, and the optical parameters of the window inside the incubator. For the latter, the side door of the incubator was replaced by a thin polyethylene cling film, which seals the incubator’s microclimate from the environment but has several orders of magnitude higher IR transparency (0.92 to 0.94) than the usual acryl (Plexiglas) doors (0.004). A compensation routine was performed with a reference measurement, which was repeated every 2 min.

Graphic Jump LocationF8 :

(a) Setup for neonatal IRT monitoring, with (b) IRT image of neonate inside an incubator, and (c) temperature development for the selected ROI.

In Fig. 8, the middle and right images are a typical freeze-frame image of an IRT sequence and the temperature development for the selected ROI. Even inside the incubator, slight differences in skin temperature can be detected. For this image the extremities are ca. 0.5°C cooler than the head and 1°C cooler than the abdomen due to good temperature insulation of the mattress and of the bent left arm.

It is interesting to take a closer look at the recordings of the infant’s nasal region. According to the breathing air flow, the surrounding skin is either cooled or heated by warm exhaled air or cooler inhaled air (Fig. 9). These temperature changes are clearly visible in the IRT recordings as long as the nasal region is in the field of view of the camera and the spatial and/or zooming ability of the camera offers sufficient image resolution of the small nasal region. Although temperature differences inside the incubator are relatively small compared to measurements on adults, with lower common environmental room temperatures of 20°C, fluctuations of 0.3°C are still visible. In the example presented in Fig. 9, the recorded RR was confirmed by additional neonatal bedside monitoring (IntelliVue MP30, Philips, Eindhoven, the Netherlands).

Graphic Jump LocationF9 :

Respiration rhythm of a neonate monitored with IRT, modified from 24.

In all recordings, the distribution of skin temperature of all not-covered skin regions could be monitored clearly. As long as the nasal regions of the neonates were visible in the IRTI image section and the neonates were calmly lying in the incubator (no turning of head or heavy struggling movements), the RRs could be calculated from analyzing the nasal region for all seven neonates at least in sections.

Both of these camera-based monitoring techniques have potential for continuous patient monitoring. They are particularly suited for neonatal care because the methods used are contactless, thus protecting sensitive skin. The feasibility result presented in this work demonstrated the high potential and beneficial effect of the steady and well-controlled environmental conditions (as in a neonatal incubator) for high-quality measurements.

PPGI and IRT complement each other by covering different vital parameters. PPGI allows one to extract HR, heart rate variability (HRV), and RR, as well as even slower rhythms around 0.1 Hz from recorded perfusion data by analyzing their frequency bands. IRT monitors the distribution and development of skin surface temperature. Also, by analyzing the nasal region, the respiration rhythm can be extracted from the changing skin temperature according to the alternating cooling and warming of skin due to exhaled and inhaled air.

Thus, a combination of both systems is desirable. To create such hybrid imaging, both camera systems must be synchronized for data recording and the lines of sight of the cameras must be defined separately. To achieve this, three basic geometric setups are differentiated (Fig. 10).

Graphic Jump LocationF10 :

The three geometric setups for hybrid imaging with IRT and PPGI. (a) Arbitrary alignment. (b) Stereoscopic view. (c) Identical perspective using a glass pane as beam splitter.

For the first setup, the positioning of the cameras can be chosen arbitrarily. Here, both cameras are usually focused on different body parts from different points of view. Therefore, it is unlikely that exactly the same body site will be selected for both recordings. However, for both systems this allows one to choose the most ideal perspective on the most relevant anatomic region separately. If selection of the same body site is required for both recordings, one of the following setups must be used instead.

The second setup is a stereoscopic-like positioning of the camera. Here, the cameras are placed close to each other with almost the same view on the object. In contrast to classical optical stereoscopy, two different camera types are used. Therefore, direct matching of identical image regions is not possible due to the different image content. Nevertheless, the two recordings are not totally different. For example, by applying block-matching algorithms on edge-filtered image data, corresponding points in both recordings can be detected. This allows a reconstruction of the recording geometrics,25,26 interpolation of the remaining image sections, and calculation of the object’s camera distance. This reconstruction process can be simplified and becomes less susceptible if the exact positioning of the camera is known or calibrated previously.

The third setup uses a beam splitter that allows measurements from the same perspective with the two systems. Though the distance information can no longer be calculated, the complex stereoscopic image processing is no longer required. A simple way of realizing a beam splitter is to use a polished glass plate. For PPGI in visible and near-infrared wavelengths, glass is almost perfectly transparent, and for IRT in the IR-C band it is sufficiently reflective. However, the glass plate and its temperature must be integrated in the IRT calibration routine. This arrangement requires simple postprocessing: first, image flipping to compensate for the mirroring effect at the glass plate and, second, image stretching and (if necessary) interpolation to compensate for different imaging properties and image resolution.

Together, both camera systems cover a broad range of monitorable vital parameters. Improvements can be achieved by utilizing redundancies, as in detection of respiration rhythms. Also, the systems are rather complementary in monitoring the effects of human thermoregulation. As shown above, PPGI allows one to detect 0.1-Hz rhythms in skin perfusion. Hence, this rhythm has its origin not in a central oscillator (like the heart or respiratory rhythm), but this effect is locally controlled. In 27 it is assigned to the vasomotor rhythms controlling heat distribution and heat transfer by rhythmically dilating and contracting different local segments of the vascular tree. It is assumed that, thereby, blood flow and its associated convective heat transfer to single body parts are regulated. In particular, this involves skin perfusion because here heat exchange with the environment as a central element in body temperature regulation takes place. Thus, a correlation between vasomotor rhythms and skin temperature can be assumed. For verification and further analysis of this phenomenon, the proposed hybrid monitoring technique is recommended. Separately, both techniques are well suited to monitor temperature or vasomotor rhythms, and both have the required spatial resolution to capture position-dependent effects. With this complementary usage of both camera systems, a new experimental access for interpretation of thermoregulatory effects is created.

Both PPGI and IRTI are contactless alternatives for long-term measurement of vital parameters. PPGI is able to detect parameters correlated with skin perfusion. Detected signals are modulated by the heartbeat allowing calculation of HR or beat-to-beat analysis, such as estimation of HRV. Besides these pulse-synchronous rhythms, the measurements contain other lower-frequency components. Analysis of the PPGI recordings and references from the literature indicate that respiratory rhythms are also extractable from the signal, as are even slower rhythms.

With IRT, skin temperature behavior, in general, can be recorded. The derived respiration rhythms can be monitored by analyzing temperature changes in the nasal region of the patient.

A combination of the two optical sensing systems (active PPGI and passive IRT) allows one to cover a broad range of vital parameters in the field of medical diagnostics and remote patient monitoring. Potential redundancies between the systems can be used to increase the stability and reliability of the obtained values. The unobtrusive, spatially resolved, and contactless technique allows a wide range of applications, even in case of highly sensitive body areas.

Lund  C. H. et al., “Disruption of barrier function in neonatal skin associated with adhesive removal,” J. Pediatr.. 131, (3 ), 367 –372 (1997). 0044-2917 CrossRef
Scalise  L., “Non contact heart monitoring,” in Advances in Electrocardiograms—Methods and Analysis. , Millis  R. M., Ed., pp. 81 –106,  InTech  (2012).
Hertzman  A. B., “The blood supply of various skin areas as estimated by the photo-electric plethysmograph,” Am. J. Physiol.. 124, (2 ), 328 –340 (1938). 0002-9513 
Hertzman  A. B., Dillon  J. B., “Applications of photoelectric plethysmography in peripheral vascular diseases,” Am. Heart J.. 20, (6 ), 750 –761 (1940). 0002-8703 CrossRef
Blazek  V., “Optoelektronische Systemkonzepte für nichtinvasive Kreislaufdiagnostik,” Opto Elektronik Magazin. 7, , 212 –219 (1991).
Goebrecht  H., Ed., Lehrbuch der Experimentalphysik, Band III Optik. ,  Walter de Gruyter ,  Berlin  (1974).
Blazek  V., Schultz-Ehrenburg  U., Quantitative Photoplethysmography. Basic Facts and Examination Tests for Evaluating Peripheral Vascular Functions. ,  VDI Verlag ,  Düsseldorf  (1996).
Schultz-Ehrenburg  U., Blazek  V., “Value of quantitative photoplethysmography for functional vascular diagnostics,” Skin Pharmacol. Appl. Skin Physiol.. 14, (5 ), 316 –323 (2001). 1422-2868 CrossRef
Perlitz  V. et al., “A self-organised rhythm in the autonomous nervous system: a preliminary interpretation of the ca. 0.15 Hz-band activity prevailing in neuronal centres and peripheral effectors,” in Computer-Aided Noninvasive Vascular Diagnostics. , Blazek  V., Schultz-Ehrenburg  U., Eds., pp. 45 –56,  Mainz Wissenschaftsverlag ,  Germany  (2005).
Schmid-Schönbein  H. et al., “Synergetic interpretation of patterned vasomotion activity in microvascular perfusion: discrete effects of myogenic and neurogenic vasoconstriction as well as arterial and venous pressure fluctuations,” Int. J. Microcirc.. 17, (6 ), 346 –359 (1997). 0167-6865 CrossRef
Venema  B. et al., “Advances in reflective oxygen saturation monitoring with a novel in-ear sensor system: results of a human hypoxia study,” IEEE Trans. Biomed. Eng.. 59, (7 ), 2003 –2010 (2012). 0018-9294 CrossRef
Wu  T., Blazek  V., Schmitt  H. J., “Photoplethysmography imaging: a new noninvasive and noncontact method for mapping of the dermal perfusion changes,” Proc. SPIE. 4163, , 62 –70 (2000). 0277-786X CrossRef
Wu  T., “PPGI: new development in noninvasive and contactless diagnosis of dermal perfusion using near infrared light,” J. GCPD e.V.. 7, (1 ), 17 –24 (2003).
Hülsbusch  M., Blazek  V., “Contactless mapping of rhythmical phenomena in tissue perfusion using PPGI,” Proc. SPIE. 4683, , 110 –117 (2002). 0277-786X CrossRef
Wieringa  F. P., Mastik  F., van der Steen  A. F. W., “Contactless multiple wavelength photoplethysmographic imaging: a first step toward ‘SpO2 camera’ technology,” Ann. Biomed. Eng.. 33, (8 ), 1034 –1041 (2005). 0090-6964 CrossRef
Hülsbusch  M., “A functional imaging technique for optoelectronic assessment of skin perfusion,” Ph.D. Thesis, RWTH Aachen University, Germany (2008).
Blazek  V., “Biomedical technology—2011 and beyond,” Commun.. 1, , 5 –12 (2011).
Planck  M., “Zur Theorie des Gesetzes der Energieverteilung im Normalspectrum,” in Verhandlungen der Deutschen physikalischen Gesellschaft. , pp. 237 –245,  Friedr. Vieweg & Sohn  (1900).
Boltzmann  L., “Ableitung des Stefan’schen Gesetzes, betreffend die Abhängigkeit der Wärmestrahlung von der Temperatur aus der electromagnetischen Lichttheorie,” Annalen der Physik und Chemie. 258, (6 ), 291 –294 (1884). 0003-3804 CrossRef
InfraTec GmbH, “Thermografie Theorie—Physikalische Grundlagen,” http://www.infratec.de/de/thermografie/thermografie-wissen/theorie.html (30  12 2013).
The Engineering Toolbox, “Emissivity Coefficients of some common Materials,” http://www.engineeringtoolbox.com/emissivity-coefficients-d_447.html (30  12 2013).
Abbas  A. K. et al., “Neonatal infrared thermography monitoring,” in Neonatal Monitoring Technologies: Design for Integrated Solutions. , Chen  W., Oetomo  S. B., Feijs  L., Eds., pp. 84 –124,  IGI Global ,  Hershey, Pennsylvania  (2012).
Abbas  A. K. et al., “Neonatal infrared thermography imaging: analysis of heat flux during different clinical scenarios,” Infrared Phys. Technol.. 55, (6 ), 538 –548 (2012). 1350-4495 
Abbas  A. K. et al., “Neonatal non-contact respiratory monitoring based on real-time infrared thermography,” Biomed. Eng. Online. 10, , 93 –109 (2011). 0141-5425 CrossRef
Longuet-Higgins  H., “A computer algorithm for reconstructing a scene from two projections,” Nature. 293, , 133 –135 (1981). 0028-0836 CrossRef
Hartley  R. I., “In defence of the eight-point algorithm,” IEEE Trans. Pattern Anal.. 19, (6 ), 580 –593 (1997). 0162-8828 CrossRef
Allen  A., “Photoplethysmography and its application in clinical physiological measurement,” Physiol Meas.. 28, (3 ), R1 –R39 (2007). 0967-3334 CrossRef

Nikolai Blanik received his Dipl.-Ing. degree in electrical engineering in 2010 from RWTH Aachen University, Aachen, Germany, where he is currently working toward his Dr.-Ing. (PhD) degree in the Philips Chair of Medical Information Technology. He also is a research assistant in Philips Chair of Medical Information Technology, RWTH Aachen University. His research interests include research of photon–tissue interaction with Monte-Carlo simulation and photoplethysmographic imaging.

Abbas K. Abbas received his MSc degree in biomedical engineering in 2004 from Al Nahrain University, Baghdad, Iraq. He is currently working toward his Dr.rer.medic. (PhD) degree in the Philips Chair of Medical Information Technology at RWTH Aachen University. His research interests include research of neonatal infrared thermography imaging and developing an intelligent monitoring solution for neonatal intensive care units.

Boudewijn Venema received his Dipl.-Ing. degree in electrical engineering in 2009 from RWTH Aachen University, Aachen, Germany, where he is currently working toward his Dr.-Ing. (PhD) degree in the Philips Chair of Medical Information Technology, where he is also a research assistant. His research interests include the study of photon–tissue interaction and long-term measurement of vital signs using photoplethysmography.

Vladimir Blazek received his Dipl.-Ing. degree in electrical engineering from the Technical University, Brno, Czech Republic, 1969, the Dr.-Ing. degree from RWTH Aachen University, Germany, 1979, and the Venia Legendi degree from the Czech Technical University, Prague, 1993. Since 2011, he is with the Philips Chair of Medical Information Technology, RWTH Aachen University. His research interests include optoelectronics in medicine, biomedical sensors, functional imaging techniques, and tissue optics.

Steffen Leonhardt received his MS degree in computer engineering from the State University of New York, Buffalo, the Dipl.-Ing. degree in electrical engineering and the Dr.-Ing. degree in control engineering both from the Technical University of Darmstadt, Germany, and the MD degree in medicine from J.W. Goethe University, Frankfurt, Germany. Since 2003, he has been a professor and head of the Chair of Medical Information Technology, RWTH Aachen University.

© The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Citation

Nikolai Blanik ; Abbas K. Abbas ; Boudewijn Venema ; Vladimir Blazek and Steffen Leonhardt
"Hybrid optical imaging technology for long-term remote monitoring of skin perfusion and temperature behavior", J. Biomed. Opt. 19(1), 016012 (Jan 17, 2014). ; http://dx.doi.org/10.1117/1.JBO.19.1.016012


Figures

Graphic Jump LocationF1 :

History of reflective photoplethysmography, from (a) Hertzman’s first optoelectronic photoplethysmograph3 to (c) modern remote photoplethysmography imaging (PPGI).16

Graphic Jump LocationF2 :

(a) Early PPGI setup realized at RWTH Aachen University in 2000.12 (b) Screen shot of a typical perfusion signal.

Graphic Jump LocationF3 :

Specific spectral radiance for black bodies at different temperatures. (a) Image shows distribution at 3000 to 6000 K. (b) Image for human body surface temperatures. For temperatures around 6000 K, the maximum intensity is radiated in the visible spectrum. For lower temperatures, this maximum decreases and shifts to the long-infrared region. The maximum spectral sensitivity of the infrared thermography camera (VarioCam hr head) lies 9.5μm in the far-infrared region19 and is therefore suitable for the study of human body temperature.

Graphic Jump LocationF4 :

Composition of detected radiation in infrared thermography (IRT) monitoring (modified from 21). The amount of radiation entering the measuring section (1) toward the IRT camera (2) is a sum of the emitted radiation (3) of the measured object (4) according to its temperature, environmental radiation (5) reflected on the object’s surface (6) and background radiation (7) transmitted through the object (8). Therefore, the total detected radiation (9) depends on the absorption and emitting characteristics of the media (10) in the measuring section.

Graphic Jump LocationF5 :

Example of PPGI analysis. (a) PPGI sequence of subject’s left hand with a small wound on the middle finger. Inside the first frame, three regions of interest (ROIs) are indicated: one inside the wound (red) and two close to each other on the back of the hand (blue and green). (b) Corresponding time signals for each ROI. (c) Band-pass filtered signals with cutoff frequencies at 0.2 and 0.6 Hz correlated with respiratory activity. (d) Band-pass filtered signals with cutoff frequencies at 0.02 and 0.2 Hz representing slow signal rhythms. (e) Fast Fourier transformation of the red signal. Five peaks in the frequency domain are visible: 0.1Hz rhythm (1), respiratory rhythm (2), heart rate (3), and harmonics of heart rate (4 and 5).

Graphic Jump LocationF6 :

Phenomenon of distributed blood volume clouds observed with PPGI on the forehead of a seated subject in a relaxed steady state. The postprocessed and colored images show perfusion-coded images taken at 14, 16, 50, and 69 s of a PPGI video sequence (Video 1, MPEG, 2 MB) [URL: http://dx.doi.org/10.1117/1.JBO.19.1.016012.1].

Graphic Jump LocationF7 :

Beat-to-beat analysis of PPG signal. (a) Raw PPG signal with points of identical phase indicated (here: maxima) of single heart beats. (b) Beat-to-beat distance of the subsequent heart beats.

Graphic Jump LocationF8 :

(a) Setup for neonatal IRT monitoring, with (b) IRT image of neonate inside an incubator, and (c) temperature development for the selected ROI.

Graphic Jump LocationF9 :

Respiration rhythm of a neonate monitored with IRT, modified from 24.

Graphic Jump LocationF10 :

The three geometric setups for hybrid imaging with IRT and PPGI. (a) Arbitrary alignment. (b) Stereoscopic view. (c) Identical perspective using a glass pane as beam splitter.

Tables

Table Grahic Jump Location
Table 1Emissivity coefficients ε in the infrared C band (8 to 14 μm).20,21

References

Lund  C. H. et al., “Disruption of barrier function in neonatal skin associated with adhesive removal,” J. Pediatr.. 131, (3 ), 367 –372 (1997). 0044-2917 CrossRef
Scalise  L., “Non contact heart monitoring,” in Advances in Electrocardiograms—Methods and Analysis. , Millis  R. M., Ed., pp. 81 –106,  InTech  (2012).
Hertzman  A. B., “The blood supply of various skin areas as estimated by the photo-electric plethysmograph,” Am. J. Physiol.. 124, (2 ), 328 –340 (1938). 0002-9513 
Hertzman  A. B., Dillon  J. B., “Applications of photoelectric plethysmography in peripheral vascular diseases,” Am. Heart J.. 20, (6 ), 750 –761 (1940). 0002-8703 CrossRef
Blazek  V., “Optoelektronische Systemkonzepte für nichtinvasive Kreislaufdiagnostik,” Opto Elektronik Magazin. 7, , 212 –219 (1991).
Goebrecht  H., Ed., Lehrbuch der Experimentalphysik, Band III Optik. ,  Walter de Gruyter ,  Berlin  (1974).
Blazek  V., Schultz-Ehrenburg  U., Quantitative Photoplethysmography. Basic Facts and Examination Tests for Evaluating Peripheral Vascular Functions. ,  VDI Verlag ,  Düsseldorf  (1996).
Schultz-Ehrenburg  U., Blazek  V., “Value of quantitative photoplethysmography for functional vascular diagnostics,” Skin Pharmacol. Appl. Skin Physiol.. 14, (5 ), 316 –323 (2001). 1422-2868 CrossRef
Perlitz  V. et al., “A self-organised rhythm in the autonomous nervous system: a preliminary interpretation of the ca. 0.15 Hz-band activity prevailing in neuronal centres and peripheral effectors,” in Computer-Aided Noninvasive Vascular Diagnostics. , Blazek  V., Schultz-Ehrenburg  U., Eds., pp. 45 –56,  Mainz Wissenschaftsverlag ,  Germany  (2005).
Schmid-Schönbein  H. et al., “Synergetic interpretation of patterned vasomotion activity in microvascular perfusion: discrete effects of myogenic and neurogenic vasoconstriction as well as arterial and venous pressure fluctuations,” Int. J. Microcirc.. 17, (6 ), 346 –359 (1997). 0167-6865 CrossRef
Venema  B. et al., “Advances in reflective oxygen saturation monitoring with a novel in-ear sensor system: results of a human hypoxia study,” IEEE Trans. Biomed. Eng.. 59, (7 ), 2003 –2010 (2012). 0018-9294 CrossRef
Wu  T., Blazek  V., Schmitt  H. J., “Photoplethysmography imaging: a new noninvasive and noncontact method for mapping of the dermal perfusion changes,” Proc. SPIE. 4163, , 62 –70 (2000). 0277-786X CrossRef
Wu  T., “PPGI: new development in noninvasive and contactless diagnosis of dermal perfusion using near infrared light,” J. GCPD e.V.. 7, (1 ), 17 –24 (2003).
Hülsbusch  M., Blazek  V., “Contactless mapping of rhythmical phenomena in tissue perfusion using PPGI,” Proc. SPIE. 4683, , 110 –117 (2002). 0277-786X CrossRef
Wieringa  F. P., Mastik  F., van der Steen  A. F. W., “Contactless multiple wavelength photoplethysmographic imaging: a first step toward ‘SpO2 camera’ technology,” Ann. Biomed. Eng.. 33, (8 ), 1034 –1041 (2005). 0090-6964 CrossRef
Hülsbusch  M., “A functional imaging technique for optoelectronic assessment of skin perfusion,” Ph.D. Thesis, RWTH Aachen University, Germany (2008).
Blazek  V., “Biomedical technology—2011 and beyond,” Commun.. 1, , 5 –12 (2011).
Planck  M., “Zur Theorie des Gesetzes der Energieverteilung im Normalspectrum,” in Verhandlungen der Deutschen physikalischen Gesellschaft. , pp. 237 –245,  Friedr. Vieweg & Sohn  (1900).
Boltzmann  L., “Ableitung des Stefan’schen Gesetzes, betreffend die Abhängigkeit der Wärmestrahlung von der Temperatur aus der electromagnetischen Lichttheorie,” Annalen der Physik und Chemie. 258, (6 ), 291 –294 (1884). 0003-3804 CrossRef
InfraTec GmbH, “Thermografie Theorie—Physikalische Grundlagen,” http://www.infratec.de/de/thermografie/thermografie-wissen/theorie.html (30  12 2013).
The Engineering Toolbox, “Emissivity Coefficients of some common Materials,” http://www.engineeringtoolbox.com/emissivity-coefficients-d_447.html (30  12 2013).
Abbas  A. K. et al., “Neonatal infrared thermography monitoring,” in Neonatal Monitoring Technologies: Design for Integrated Solutions. , Chen  W., Oetomo  S. B., Feijs  L., Eds., pp. 84 –124,  IGI Global ,  Hershey, Pennsylvania  (2012).
Abbas  A. K. et al., “Neonatal infrared thermography imaging: analysis of heat flux during different clinical scenarios,” Infrared Phys. Technol.. 55, (6 ), 538 –548 (2012). 1350-4495 
Abbas  A. K. et al., “Neonatal non-contact respiratory monitoring based on real-time infrared thermography,” Biomed. Eng. Online. 10, , 93 –109 (2011). 0141-5425 CrossRef
Longuet-Higgins  H., “A computer algorithm for reconstructing a scene from two projections,” Nature. 293, , 133 –135 (1981). 0028-0836 CrossRef
Hartley  R. I., “In defence of the eight-point algorithm,” IEEE Trans. Pattern Anal.. 19, (6 ), 580 –593 (1997). 0162-8828 CrossRef
Allen  A., “Photoplethysmography and its application in clinical physiological measurement,” Physiol Meas.. 28, (3 ), R1 –R39 (2007). 0967-3334 CrossRef

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Topic Collections

Advertisement
  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.