Snapshot RGB mapping of skin melanin and hemoglobin

Janis Spigulis; Ilze Oshina

doi:10.1117/1.JBO.20.5.050503

20 May 2015 Snapshot RGB mapping of skin melanin and hemoglobin

Janis Spigulis, Ilze Oshina

Author Affiliations +

Journal of Biomedical Optics, Vol. 20, Issue 5, 050503 (May 2015). https://doi.org/10.1117/1.JBO.20.5.050503

Abstract

The concept of snapshot red-green-blue (RGB) multispectral imaging was applied for skin chromophore mapping. Three monochromatic spectral images have been extracted from a single RGB image dataset at simultaneous illumination of skin by 473-, 532-, and 659-nm laser lines. The spectral images were further transformed into distribution maps of skin melanin, oxyhemoglobin, and deoxyhemoglobin, related to pigmented and vascular skin malformations. The performance and clinical potential of the proposed technique are discussed.

1. Introduction

Fast imaging of spectral reflectance is an important issue for a range of application areas including skin chromophore mapping for express diagnostics, recovery monitoring, forensic evaluation, and so on.¹^–⁶ The shorter the image acquisition time, the lower is the probability of motion artifacts to be corrected afterward. The narrower are the selected spectral bands, the higher is the performance and reliability of spectral imaging and the subsequent parametric mapping. Consequently, single snapshot monochromatic spectral imaging at several fixed wavelengths represents a promising technique for fast and reliable parametric imaging of skin.

Snapshot narrowband spectral imaging by means of additional dispersive elements in front of the image sensors has been reported previously.⁷^,⁸ This approach, however, is costly and excludes direct use of conventional red-green-blue (RGB) cameras, e.g., those embedded in PCs and smartphones. Recently, colorimetric analysis of human tissues showed promising results.⁹ RGB snapshot chromophore mapping based on skin color analysis without any additional hardware was also demonstrated.¹⁰ The complicated processing procedure (conversion of color coordinates, solution of integral equations comprising skin chromophore absorption spectra), being time/resource consuming and very sensitive to the measurement and calculation errors, is the main drawback of this approach. Therefore, alternative solutions excluding the need for colorimetric conversions are of interest for technicians and clinicians.

Extraction of several spectral images from a single RGB image dataset by exploiting the spectral features of the image sensor and/or spectrally specific illumination of the target was proposed and discussed recently.¹¹^–¹⁵ Snapshot RGB mapping of skin hemoglobin at dual-wavelength laser illumination has been demonstrated;¹¹ performance of this technique can be further improved by the corresponding RGB band crosstalk corrections.¹³ Uniform illumination simultaneously at three fixed wavelengths allows extracting three monochromatic spectral images from a single RGB image dataset.¹⁵ In this case, solutions of relatively simple linear equations may ensure fast image processing and mapping of three main skin chromophores. An experimental study aiming to check the viability of this concept is described below.

2. Methods

2.1.

Concept

The presented approach exploits only the spectral features of the image sensor R, G, and B detection bands, avoiding any transformations to the color systems like L*a*b* or RGB gamut. Basic principles of the proposed technique were described in detail earlier.¹⁵ Generally, if the target illumination spectrum comprises only a limited number of narrow spectral lines with fixed wavelengths, monochromatic spectral images at these wavelengths can be extracted from a single snapshot RGB image dataset by separate registration of the R, G, and B output values from each image pixel or specified pixel group. Subsequent mapping of the spectral reflectance at each of the illumination wavelengths is possible this way if the following preconditions are met: (1) the RGB spectral sensitivity curves of the image sensor are known—given by the manufacturer or directly measured, (2) they are uniform—the same for all pixels over the whole area of the RGB image sensor, (3) linear photovoltaic responses at all three detection channels (R, G, and B) are ensured, (4) the image field comprises a reference reflector, related to a particular pixel group of the image.

2.2.

Measurement Setup

The measurement setup comprised an objective-supplied CMOS camera (USB 2 UI-1226LE-C, IDS with known RGB sensitivity curves¹³^,¹⁶) and three stabilized continuous wave laser modules [473 nm, MBL-III-473; 532 nm, DD532-10-5; and 659 nm, DB650-12-3(5)] with comparable output powers in the range 10 to 15 mW. Each laser output beam was launched into a bundle of seven optical fibers with silica core diameter of 400 μm, mounted within the SMA-standard connector; their outputs formed an illumination ring, composed of 21 randomly distributed emitting fiber ends around the camera objective. Each laser module could be independently switched on and off. The target plane was located 50 mm from the CMOS objective lens. Uniformity of target illumination was enhanced by a toroidal lens in front of the emitter ring. Polarizing film was placed behind this lens, a film with orthogonal polarization direction—in front of the camera objective, in order to minimize the specular reflection from the target. Laser output powers and the exposition time of the CMOS sensor were carefully adjusted to provide linearity of the photoresponse at all spectral combinations used in the experiments. All automatic features of the CMOS image sensor were switched off. A flat paper target composed of four color segments (red, green, blue, and white) was used for primary calibration of the system as in the previous study.¹⁵ The colors were selected to ensure a unique spectral image pattern at each illumination wavelength; that allowed comparing the “true” spectral images (under illumination of a single laser line) with those extracted from the RGB image dataset under simultaneous illumination by three different laser lines. The illumination and recording conditions were optimized with respect to such comparisons. After that, the color target was replaced by healthy or pathological in vivo skin areas in order to map the distributions of three skin chromophores. Altogether 10 skin malformations (5 pigmented and 5 vascular, all confirmed by dermatologist) were examined under approval of the local ethics committee.

2.3.

Image Processing

Mapping of skin spectral reflectance at the three chosen wavelengths was the first goal of this study. If preconditions (1)–(4) are met, relatively simple linear equations describe the RGB digital output signals of the image sensor.¹⁵ The main variables influencing the results are ratios of the spectral sensitivities of each detection band (R, G, and B) at the three defined wavelengths and the measured output signals of the R-, G-, and B-channel from the pixels located in the target zone and in the reference zone. Areas of pigmented and vascular skin malformations were chosen to be the target zones in this study, while the adjacent skin areas served as reference zones. As a result, RGB crosstalk-corrected images of relative spectral reflectance at the three specified wavelengths were constructed.

The second goal was to convert the obtained set of three monochromatic images into distribution maps of skin melanin, oxyhemoglobin, and deoxyhemoglobin. The three-chromophore skin model⁴^,⁵ based on the Beer–Lambert law leads to the following system of three equations to be applied for each $x - y$ pixel of the three spectral images:

Eq. (1)

{\begin{cases} c_{a} \cdot ε_{a} (λ_{1}) + c_{b} \cdot ε_{b} (λ_{1}) + c_{c} \cdot ε_{c} (λ_{1}) = - \frac{\ln k_{1}}{l_{1}} \\ c_{a} \cdot ε_{a} (λ_{2}) + c_{b} \cdot ε_{b} (λ_{2}) + c_{c} \cdot ε_{c} (λ_{2}) = - \frac{\ln k_{2}}{l_{2}} \\ c_{a} \cdot ε_{a} (λ_{3}) + c_{b} \cdot ε_{b} (λ_{3}) + c_{c} \cdot ε_{c} (λ_{3}) = - \frac{\ln k_{3}}{l_{3}} \end{cases},

where

k_{i}

is the relative spectral reflectance at the wavelength

λ_{i}

(

λ_{1} = 473 nm

,

λ_{2} = 532 nm

,

λ_{3} = 659 nm

),

l_{i}

is the mean optical path length at the particular wavelength,

ε (λ_{i})

is the extinction coefficient of the specified chromophore (

a

—oxyhemoglobin,

b

—deoxyhemoglobin, and

c

—melanin) at the

i

wavelength, and

c_{j}

is the relative concentration of the chromophore to be mapped (

j = a

,

b

, or

c

).

Tabulated values of the extinction coefficients of hemoglobin¹⁷ and melanin¹⁸ at the three specified wavelengths were used in calculations. As for the absorption path lengths, several options have been considered with respect to skin anatomy and different mean light penetration depths $δ$ in the skin.¹⁹ The 473-nm radiation ( $δ = 190 μ m$ ) can be absorbed mainly by skin melanin within the epidermal layer of thickness $\sim 150$ to $200 μ m$ ,²⁰ but also some absorption by the hemoglobin of upper dermal blood vessels has to be taken into account at this wavelength. The two other exploited wavelengths penetrated deeper (532 nm, $δ = 330 μ m$ and 659 nm, $δ = 660 μ m$ ) and could be more notably absorbed by dermal hemoglobin. Our analysis showed that reasonable results can be obtained with an estimated path length $4 δ$ in all cases except for the 473-nm absorption by hemoglobin, where $l = 2 δ$ appeared to be a good approximation.

The solution of Eq. (1) for each image pixel at the above-mentioned conditions and embedded numerical data comprising the nine extinction coefficients¹⁷^,¹⁸ is the following:

Eq. (2)

c_{a} = 0.145 \cdot \ln k_{3} - 0.033 \cdot \ln k_{2} - 0.024 \cdot \ln k_{1},

Eq. (3)

c_{b} = - 0.128 \cdot \ln k_{3} - 0.024 \cdot \ln k_{2} + 0.057 \cdot \ln k_{1},

Eq. (4)

c_{c} = - 0.738 \cdot \ln k_{3} + 0.408 \cdot \ln k_{2} - 0.806 \cdot \ln k_{1} .

Equations (2) to (4) were further used for mapping skin oxyhemoglobin, deoxyhemoglobin, and melanin, respectively.

To summarize, the image processing comprised the following steps:

1. Splitting the original color image into its three RGB channels.
2. Extracting the background normal skin area and calculating the mean value for each RGB channel.
3. Extracting the region of interest.
4. Calculating the mean values of the normal skin at the three specified wavelengths by using the RGB crosstalk correction algorithm.¹⁵
5. Calculating the values of relative spectral reflectances.
6. Calculating the chromophore concentrations in every pixel of the image using Eqs. (2) to (4) to compose the chromophore maps.

3. Results and Discussion

The obtained chromophore maps for the same malformation type (pigmented or vascular) showed similar features, therefore, only one typical set of chromophore maps for each malformation type is illustrated in Fig. 1. Units of the color bars are $10^{- 2} • mol / L$ .

Fig. 1

(a) Skin image at trichromatic laser illumination and (b) distribution maps of three skin chromophores for vascular (upper row) and pigmented (lower row) skin malformations. Scale bar on (a) 1 cm.

The obtained chromophore maps are in qualitative agreement with the physiologically expected distributions of chromophores in the regarded skin malformations. In the case of vascular pathology (upper row), blood oxyhemoglobin content in the malformation region has increased (lighter spot) due to additional supply of arterial blood in the damaged dermal zone, deoxyhemoglobin content correspondingly decreased compared with the adjacent skin (darker spot), and melanin content in the epidermal layer remained practically unchanged. In the case of pigmented pathology (lower row), a pronounced increase of epidermal melanin content (lighter spot) is observed without significant changes in the oxy- and deoxyhemoglobin contents in the dermal blood vessels, as one could expect.

The color scales represent the relative increase or decrease of the chromophore concentrations with respect to their mean values in the surrounding healthy skin. Eventually, quantitative estimation of chromophore concentrations may also become possible after cross-validation with calibrated skin phantoms. Meanwhile, presentation of chromophore maps in relative units with respect to healthy skin of the same body region seems to be better adapted for clinical use (especially for early-stage assessment and recovery monitoring), if compared with approximate estimation of absolute concentrations of chromophores. Even if reliable reference phantoms become available, in reality, the reflectance calibration results may appear mistaken due to the large variety of skin colors and anatomic features, e.g., different thicknesses of the melanin-containing epidermis at different sites of the body.

The character of chromophore distribution at the surrounding skin tissue in both cases is rather uneven or grainy. At the current stage, it is not clear how far it reflects the real distribution of the particular chromophores and how far it is influenced by the laser illumination speckles. Nevertheless, the obtained contrast at the malformation regions was high enough to reliably separate healthy and pathologic skin areas in the chromophore maps. To summarize, the proposed noncontact express-technique adequately reflected skin chromophore distributions and may find clinical applications for primary diagnostics of skin lesions, as well as for monitoring of skin recovery processes after surgeries, burns, bruises, and other interventions.

4. Conclusions

The obtained results demonstrated adequate mapping of skin melanin and hemoglobin by the single-shot RGB technique, thus confirming viability of the previously proposed concept.¹⁵ To the author’s knowledge, this was the first experimental validation of skin chromophore snapshot RGB mapping under triple wavelength illumination. Clear advantages of this approach are the simplicity of the procedure (single snapshot), wide access of suitable consumer RGB cameras (e.g., smartphone cameras) and fast image processing by means of linear polynomials [Eqs. (2)–(4)] comprising three measurable parameters per pixel. The results also highlighted the ways for further improvements. Main items to be solved in future are calibration of the chromophore content in distribution maps and development of handy triple-wavelength light sources ensuring uniform skin illumination. A more sophisticated three laser illumination system design is in progress, as well as an alternative speckle-free illumination system.

Acknowledgments

This work was supported by the European Regional Development Fund project “Innovative technologies for optical skin diagnostics” (#2014/0041/2DP/2.1.1.1.0/14/APIA/VIAA/015).

References

1.

L. Randeberg, E. Larsen and L. Svaasand, “Characterization of vascular structures and skin bruises using hyperspectral imaging, image analysis and diffusion theory,” J. Biophotonics, 3 53 –65 (2010). http://dx.doi.org/10.1002/jbio.200910059 JBOIBX 1864-063X Google Scholar

2.

D. Yudovsky et al., “Assessing diabetic foot ulcer development risk with hyperspectral tissue oximetry,” J. Biomed. Opt., 16 026009 (2011). http://dx.doi.org/10.1117/1.3535592 JBOPFO 1083-3668 Google Scholar

3.

G. N. Stamatas and N. Kollias, “In vivo documentation of cutaneous inflammation using spectral imaging,” J. Biomed. Opt., 12 051603 (2007). http://dx.doi.org/10.1117/1.2798704 JBOPFO 1083-3668 Google Scholar

4.

D. Jakovels and J. Spigulis, “2-D mapping of skin chromophores in the spectral range 500–700 nm,” J. Biophotonics, 3 125 –129 (2010). http://dx.doi.org/10.1002/jbio.v3:3 JBOIBX 1864-063X Google Scholar

5.

I. Kuzmina et al., “Towards noncontact skin melanoma selection by multispectral imaging analysis,” J. Biomed. Opt., 16 060502 (2011). http://dx.doi.org/10.1117/1.3584846 JBOPFO 1083-3668 Google Scholar

6.

D. Jakovels, U. Rubins and J. Spigulis, “RGB imaging system for mapping and monitoring of hemoglobin distribution in skin,” Proc. SPIE, 8158 81580R (2011). http://dx.doi.org/10.1117/12.893789 PSISDG 0277-786X Google Scholar

7.

L. Kong et al., “Single sensor that outputs narrowband multispectral images,” J. Biomed. Opt., 15 010502 (2010). http://dx.doi.org/10.1117/1.3277669 JBOPFO 1083-3668 Google Scholar

8.

W. R. Johnson et al., “Snapshot hyperspectral imaging in ophthalmology,” J. Biomed. Opt., 12 014036 (2007). http://dx.doi.org/10.1117/1.2434950 JBOPFO 1083-3668 Google Scholar

9.

G. I. Petrov et al., “Human tissue color as viewed in high dynamic range optical transmission measurements,” Biomed. Opt. Express, 3 (9), 2154 –2161 (2012). http://dx.doi.org/10.1364/BOE.3.002154 BOEICL 2156-7085 Google Scholar

10.

I. Nishidate et al., “Visualizing of skin chromophore concentrations by use of RGB images,” Opt. Lett., 33 (19), 2263 –2271 (2008). http://dx.doi.org/10.1364/OL.33.002263 OPLEDP 0146-9592 Google Scholar

11.

J. Spigulis, D. Jakovels and U. Rubins, “Multi-spectral skin imaging by a consumer photo-camera,” Proc. SPIE, 7557 75570M (2010). http://dx.doi.org/10.1117/12.845492 PSISDG 0277-786X Google Scholar

12.

J. Spigulis, D. Jakovels and U. Rubins, “Method and device for multi-spectral imaging by means of a digital RGB sensor,” WO 2012/002787 A1 (2012).

13.

J. Spigulis, D. Jakovels and L. Elste, “Towards single snapshot multispectral skin assessment,” Proc. SPIE, 8216 82160L (2012). http://dx.doi.org/10.1117/12.908967 PSISDG 0277-786X Google Scholar

14.

J. Spigulis and L. Elste, “Method and device for imaging of spectral reflectance at several wavelength bands,” WO 2013135311 A1 (2012).

15.

J. Spigulis and L. Elste, “Single snapshot RGB multispectral imaging at fixed wavelengths: proof of concept,” Proc. SPIE, 8937 89370L (2014). http://dx.doi.org/10.1117/12.2039442 PSISDG 0277-786X Google Scholar

16.

IDS Imaging Development Systems GmbH, “Technical data of CMOS camera model USB 2 UI-122 6LE-C,” (2015) http://www.ids-imaging.de/frontend/products.php?cam_id=12 May ). 2015). Google Scholar

17.

S. Prahl, “Tabulated molar extinction coefficient for hemoglobin in water,” (2015) http://omlc.ogi.edu/spectra/hemoglobin/summary.html May ). 2015). Google Scholar

18.

T. Sarna and H. M. Swartz, “The physical properties of melanin,” (2015) http://omlc.ogi.edu/spectra/melanin/eumelanin.html May ). 2015). Google Scholar

19.

R. R. Anderson and J. A. Parrish, “The optics of human skin,” J. Invest. Dermatol., 77 (1), 13 –19 (1981). http://dx.doi.org/10.1111/jid.1981.77.issue-1 JIDEAE 0022-202X Google Scholar

20.

K. S. Stenn, “The skin,” Cell and Tissue Biology, 541 –572 Urban & Shwarzenberg, Baltimore (1988). Google Scholar

Citation Download Citation

Janis Spigulis and Ilze Oshina "Snapshot RGB mapping of skin melanin and hemoglobin," Journal of Biomedical Optics 20(5), 050503 (20 May 2015). https://doi.org/10.1117/1.JBO.20.5.050503

Published: 20 May 2015

Access the abstract

JOURNAL ARTICLE
3 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

CITATIONS

Cited by 17 scholarly publications and 2 patents.

Explore citations on Lens.org

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Skin

RGB color model

Chromophores

Image sensors

Reflectivity

Cameras

Absorption

1.

Introduction

2.

Methods

2.1.

Concept

2.2.

Measurement Setup

2.3.

Image Processing

Eq. (1)

Eq. (2)

Eq. (3)

Eq. (4)

3.

Results and Discussion

Fig. 1

4.

Conclusions

Acknowledgments

References

Show All Keywords

Keywords/Phrases

Search In:

Publication Years