Special Section on Optical Diagnostic and Biophotonic Methods from Bench to Bedside

Image enhancement based on in vivo hyperspectral gastroscopic images: a case study

[+] Author Affiliations
Xiaozhou Gu, Zhimin Han, Ye Fu, Changsheng Liu, Xiguang Wang, Tianyu Xie

Peking University, Department of Biomedical Engineering, College of Engineering, Liaokaiyuan Building, Room 2-301, Haidian, Beijing 100871, China

Liqing Yao, Yunshi Zhong, Qiang Shi

Zhongshan Hospital, Endoscopy Center, 180 Fenglin Road, Xuhui, Shanghai 200032, China

J. Biomed. Opt. 21(10), 101412 (May 20, 2016). doi:10.1117/1.JBO.21.10.101412
History: Received October 23, 2015; Accepted April 28, 2016
Text Size: A A A

Open Access Open Access

Abstract.  Hyperspectral imaging (HSI) has been recognized as a powerful tool for noninvasive disease detection in the gastrointestinal field. However, most of the studies on HSI in this field have involved ex vivo biopsies or resected tissues. We proposed an image enhancement method based on in vivo hyperspectral gastroscopic images. First, we developed a flexible gastroscopy system capable of obtaining in vivo hyperspectral images of different types of stomach disease mucosa. Then, depending on a specific object, an appropriate band selection algorithm based on dependence of information was employed to determine a subset of spectral bands that would yield useful spatial information. Finally, these bands were assigned to be the color components of an enhanced image of the object. A gastric ulcer case study demonstrated that our method yields higher color tone contrast, which enhanced the displays of the gastric ulcer regions, and that it will be valuable in clinical applications.

Figures in this Article

Spectral measurements and analyses, which could provide accurate quantifications of the microvascular and morphological properties of gastric mucosa, facilitate the diagnosis of gastric diseases. However, point spectroscopy does not account for the spatial heterogeneity of tissue, as it detects light from a single point, which makes it ineffective for mapping lesion areas. Hyperspectral imaging (HSI), on the other hand, can be used for spatial mapping of tissue morphology and physiology,17 and it accounts for the spatial heterogeneity of the tissue. This technique involves the acquisition and analysis of a series of reflected two-dimensional images sampled at different wavelengths.813

In the gastrointestinal field, most HSI studies have involved ex vivo biopsies, resected tumor tissues, or surface organs, such as the skin, tongue, or larynx. Clancy et al.13 developed a laparoscope HSI system based on a liquid-crystal tunable filter, Martin et al.11,12 designed an HSI system that employs fluorescence for larynx imaging, and Leitner et al.14 established an HSI laryngoscope system based on an acoustic-optical tunable filter, which can collect 51 images with a bandwidth of 5 nm in 1.25 s. Kiyotoki et al.15 proposed the use of an HSI camera in a diagnostic support system for resected gastric cancer tissues, and indicated that HSI could be employed to measure spectral reflectance in gastric tumors and could differentiate between tumorous and normal mucosa. However, the colors of the resected tissues differed from those observed in vivo, which suggests that the spectral properties of tissue may change after resection.

In the present study, we developed an image enhancement method based on in vivo hyperspectral images for the examination of different types of gastric diseases.

Equipment Setup

Figure 1 shows a diagram of the flexible hyperspectral (HS) gastroscopy system, which can be used to obtain a series of reflected HS images in a noncontact manner in the 405 to 665 nm wavelength range. This system is a modified version of the commercialized endoscope AQ-100 donated by Shanghai Aohua Photoelectric Endoscope Co. Ltd. This system is described in detail in Ref. 16. Twenty-seven sequential narrowband bandpass interference filters are centered at wavelengths between 405 and 665 nm in 10 nm intervals with full widths at half maximum (FWHMs) of 10 nm and 75% peak transmission (Shenyang HB Optical Technology Co., Ltd., China) and, together with two all-pass holes, are mounted in two motorized filter wheels positioned in the collimated light beam path. In the conventional white light illumination mode, red (R), green (G), and blue (B) color filters are switched and the all-pass holes in the HS wheels are in the illumination beam path. When the suspected disease area is targeted and the HSI mode is enabled, the 27 monochromatic spectral-channel light beams are produced sequentially in 4.2 s as the filter wheel rotates. An optical fiber bundle directs the beams to the mucosa after the optical taper focuses the light onto the bundle. A combination of lenses is attached to the distal end of the bundle. This combination results in a 5.0-cm-diameter illumination spot at a working distance of 2.0 cm. A 582×752-pixel monochromatic charge-coupled device (CCD; ICX279AL, Sony) positioned at the distal end of the optical fiber bundle is used to spectrally resolve the reflected images from the mucosa. Analog signals read from the CCD are sequentially transformed into digital signals in the image processing unit. A cable connects the image processing unit to the medical monitor, which displays the reflected images of the mucosa.

Graphic Jump Location
Fig. 1
F1 :

Diagram of flexible HSI gastroscopy system.

Clinical Data Acquisition

After obtaining ethics approval from Zhongshan Hospital (Shanghai, China, Approval No. 2012-06(2)) and informed consent from the patients, we acquired HS images of different types of gastric mucosa in a clinical setting. In particular, HS images of gastric ulcers were investigated.

Data Preprocessing

After acquiring sequential HS images containing different spectral information, we performed image preprocessing to reduce the imperfections that arose during imaging and to generate images suitable for analysis. Noise reduction, contrast enhancement, illumination normalization, reflex removal, and soft image registration for moving objects were performed.1722

Band Selection

The entropy of an image is defined as23,24Display Formula

H(A)=  p(A)logp(A),(1)
where H is entropy, A is the pixel variable of the image, and p is the probability distribution function of A. The joint entropy is defined as23,24Display Formula
I(A,B)=p(A,B)logp(A,B)p(A)p(B),(2)
where I indicates the joint entropy, A and B represent the pixel variables of two different spectral images, and p(A,B) is the joint probability distribution function of A and B. The joint entropy can also be represented as Display Formula
I(A,B)=H(A)+H(B)H(A,B).(3)

Then the dependence of information (DI) is defined as23,24Display Formula

θDI=H(A1,,An)i1=1nH(Ai1|Ai2,,Ain),(4)
where θDI is the DI index and Display Formula
H(A1,,An)=p(a1,,an)log2p(a1,,an)(5)
and Display Formula
H(A|B)=H(A,B)H(B).(6)

The value of θDI can be used to identify the relevant bands’ selection in HS image cubes by performing the following procedure:23

  • Calculate the entropy for each image in the image cube, and make the image with the highest entropy be the first main component W1.
  • Calculate the mutual information (MI) between W1 and the other images in the image cube, and make the image with the lowest MI be the second main component W2.
  • Calculate θDI for W1 and W2, and the other images in the image cube, make the image that has the minimum θDI be the third main component W3.
  • Continue step 3 until enough components have been selected.

Image Enhancement and Evaluation

After selecting the band subset in this study, the selected bands were assigned to be the new R, G, and B components for the enhanced image. After performing white balancing, the effects of this image enhancement method were investigated by comparing the new image with the conventional color image.

After developing the system, we obtained 29 in vivo HS image cubes for normal gastric mucosa tissues and different types of gastric disease tissues and objects from 12 volunteers in a clinic. Figure 2 shows example images from an HS image cube for a gastric ulcer. The spectral characteristics in the ulcer region are significantly different from those of the normal tissue and the erythematous mucosa around the disease area. Figure 3 presents the spectral curves from three typical regions in the image. As shown, the ulcer region has a reflectance higher than those of the erythematous and normal regions at all visible wavelengths. This increased reflectance is related to the fact that the ulcer region is whiter than its surroundings. Below 595 nm, the reflectance of the erythematous region is lower than that of the normal tissue regions, while above 595 nm, their reflectances are similar. Hence, the erythematous region appears redder than the normal region. We used this gastric ulcer HS image cube in the image enhancement case study.

Graphic Jump Location
Fig. 2
F2 :

First image is conventional color image, while others are images from HS image cube for gastric ulcer, individually centered at 465, 515, 545, 575, 595, 615, and 665 nm.

Graphic Jump Location
Fig. 3
F3 :

Spectral curves from representative points in HS image cube. There are three typical regions: ulcer region (gastrohelcosis), erythematous region, and normal tissue region.

Table 1 presents the band selection results obtained using the DI algorithm, which shows that the first three selected bands are from the G, B, and R wavelength regions. These three bands were assigned to be the new R, G, and B components for the enhanced image. We also assigned the 465 nm band to be the B component for comparison.

Table Grahic Jump Location
Table 1Band selection results obtained using DI algorithm on HS images of gastric ulcer.

Figure 4 shows the enhanced imaging results obtained using the new assignments. Figures 4(a) and 4(b) show that the ulcer region appears yellower in the enhanced image than in the conventional image, while the normal tissue regions appear whiter. The erythematous region located between the ulcer region and the normal region appears similar in the enhanced and conventional images. Thus, the enhanced image facilitates differentiation between the ulcer and erythematous regions from the normal tissue region.

Graphic Jump Location
Fig. 4
F4 :

(a) Gastric ulcer region in conventional color imaging mode. (b) Enhanced image using HS images at 645, 545, and 495 nm as new R, G, and B components, respectively. (c) Enhanced image using HS images at 645, 545, and 465 nm as new R, G, and B components, respectively, (b) and (c) were obtained after white balance processing.

From Figs. 4(b) and 4(c), it is evident that the erythematous and normal regions have similar color tones, while the ulcer region exhibits a different color. This finding indicates that different band selections could produce different color tones and enhanced results.

For a gastroscopy system with CCDs operating at low speeds, the use of more than three bands would prevent real-time imaging. Hence, three bands should be employed to facilitate diagnosis, as the bands would be selected according to the specific disease and would have FWHMs narrower than those in conventional white light imaging and narrowband imaging.

Despite the interesting results obtained in this study, this method should be employed clinically in the future to verify its effectiveness in actual application. Moreover, since many factors, such as the organ, location, disease stage, and environment, may cause the reflected spectral images to vary, we do not expect that these band selection results will be universally applicable. In this study, we simply proved that this image enhancement method based on in vivo HS images is useful and that it can improve the color tone contrast of images used for disease diagnosis.

In the future, after analyzing a large number of cases involving in vivo HSI application and the bands selected for each disease and individual, we will be able to determine the optimal spectral band subsets that are effective for most patients with specific diseases.

We proposed and demonstrated an image-enhancement method based on band selection for in vivo HS gastroscopic imaging. A flexible gastroscopy system was modified with HS technology and was used to obtain in vivo HS images for different types of mucosal diseases and objects. A band selection algorithm based on DI was employed to analyze HS images of a gastric ulcer, and three bands (645, 545, and 495 nm) were identified from these images. After defining these selected bands as the new R, G, and B components to be used to obtain the enhanced image, it was found that this new method yielded color tone contrast higher than that of the corresponding conventional color image. Although further study is required, this method has great potential for application in clinics.

The authors were grateful to engineers Mingda Chen and Zongxiao Sun from Peking University.

Wolfe  W. L., Introduction to Imaging Spectrometers. ,  SPIE Press ,  Bellingham, Washington  (1997).
Lu  G. L., and Fei  B. W., “Medical hyperspectral imaging: a review,” J. Biomed. Opt.. 19, (1 ), 010901  (2014).CrossRef
Wang  L. V., Photoacoustic Imaging and Spectroscopy. ,  CRC Press ,  Boca Raton, Florida  (2009).
Boas  D. A., , Pitris,  C., and Ramanujam  N., Handbook of Biomedical Optics. ,  CRC Press ,  Boca Raton, Florida  (2012).
Tuchin  V. V., and Tuchin  V., Tissue Optics: Light Scattering Methods and Instruments for Medical Diagnosis. ,  SPIE Press ,  Bellingham, Washington  (2007).
Pierce  M. C.  et al., “Accuracy of in vivo multimodal optical imaging for detection of oral neoplasia,” Cancer Prev. Res.. 5, (6 ), 801 –809 (2012).CrossRef
Akbari  H.  et al., “Hyperspectral imaging and quantitative analysis for prostate cancer detection,” J. Biomed. Opt.. 17, (7 ), 076005  (2012).CrossRef
Masood  K.  et al., “Hyperspectral colon tissue classification using morphological analysis,” in  Proc. Second Int. Conf. on Emerging Technologies 2006 , pp. 735 –741 (2006).
Lindsley  E., , Wachman  E. S., and Farkas  D. L., “The hyperspectral imaging endoscope: a new tool for in vivo cancer detection,” Proc. SPIE. 5322, , 75 –82 (2004).CrossRef
Gerstner  A. O. H.  et al., “Hyperspectral imaging of mucosal surfaces in patients,” J. Biophotonics. 5, (3 ), 255 –262 (2012).CrossRef
Martin  M. E.  et al., “Development of an advanced hyperspectral imaging (HSI) system with applications for cancer detection,” Ann. Biomed. Eng.. 34, (6 ), 1061 –1068 (2006).CrossRef
Martin  R., , Thies  B., and Gerstner  A. O. H., “Hyperspectral hybrid method classification for detecting altered mucosa of the human larynx,” Int. J. Health Geographics. 11, , 21  (2012).
Clancy  N. T.  et al., “Multispectral image alignment using a three channel endoscope in vivo during minimally invasive surgery,” Biomed. Opt. Express. 3, (10 ), 2567 –2578 (2012).CrossRef
Leitner  R.  et al., “Multi-spectral video endoscopy system for the detection of cancerous tissue,” Pattern Recognit. Lett.. 34, (1 ), 85 –93 (2013)CrossRef
Kiyotoki  S.  et al., “New method for detection of gastric cancer by hyperspectral imaging: a pilot study,” J. Biomed. Opt.. 18, (2 ), 026010  (2013)CrossRef
Han  Z.  et al., “In vivo use of hyperspectral imaging to develop a noncontact endoscopic diagnosis support system for malignant colorectal tumors,” J. Biomed. Opt.. 21, (1 ), 016001  (2016)CrossRef
Chaudhuri  S.  et al., “Detection of blood-vessels in retinal images using two-dimensional matched-filters,” IEEE Trans. Med. Imaging. 8, (3 ), 263 –269 (1989)CrossRef
Crum  W. R., , Hartkens  T., and Hill  D. L. G., “Non-rigid image registration: theory and practice,” Br. J. Radiol.. 77, , S140 –S153 (2004).CrossRef
Kybic  J., “Elastic image registration using parametric deformation models,” Citeseer. , PhD dissertation, 13 –159 (2001).
Lange  H.  et al., “Reflectance and fluorescence hyperspectral elastic image registration,” Proc. SPIE. 5370, , 11  (2004).CrossRef
Mitra  R. S., and Murthy  N. N., “Elastic, maximal matching,” Pattern Recognit.. 24, (8 ), 747 –753 (1991).CrossRef
Zitova  B., and Flusser  J., “Image registration methods: a survey,” Image Vision Comput.. 21, (11 ), 977 –1000 (2003).CrossRef
Guo  B. F.  et al., “Band, selection for hyperspectral image classification using mutual information,” IEEE Geosci. Remote Sens. Lett.. 3, (4 ), 522 –526 (2006).CrossRef
Sotoca  J. M., , Pla  F., and Klaren  A. C., “Unsupervised band selection for multispectral images using information theory,” in  Proc. of the 17th Int. Conf. on Pattern Recognition , Vol. 3, , pp. 510 –513 (2004).

Xiaozhou Gu is a PhD student in the Department of Biomedical Engineering at Peking University. His research interests include medical device analysis and marketing.

Zhimin Han is a postdoc in the College of Engineering at Peking University. His research interests include medical device development, electronics hardware design, and medical image analysis.

Liqing Yao is a gastrointestinal endoscopy doctor at Zhongshan Hospital in Shanghai, China.

Yunshi Zhong is a gastrointestinal endoscopy doctor at Zhongshan Hospital in Shanghai, China.

Qiang Shi is a gastrointestinal endoscopy doctor at Zhongshan Hospital in Shanghai, China.

Ye Fu is a master’s student in the Department of Biomedical Engineering at Peking University. His research interests include medical device development and electronics hardware design.

Changsheng Liu is a PhD student in the Department of Biomedical Engineering at Peking University.

Xiguang Wang is a PhD student in the Department of Biomedical Engineering at Peking University. His research interests include medical device development and electronics hardware design.

Tianyu Xie is a professor in the Department of Biomedical Engineering at Peking University. His research interests include medical device development and biomedical optics.

© The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Citation

Xiaozhou Gu ; Zhimin Han ; Liqing Yao ; Yunshi Zhong ; Qiang Shi, et al.
"Image enhancement based on in vivo hyperspectral gastroscopic images: a case study", J. Biomed. Opt. 21(10), 101412 (May 20, 2016). ; http://dx.doi.org/10.1117/1.JBO.21.10.101412


Figures

Graphic Jump Location
Fig. 1
F1 :

Diagram of flexible HSI gastroscopy system.

Graphic Jump Location
Fig. 2
F2 :

First image is conventional color image, while others are images from HS image cube for gastric ulcer, individually centered at 465, 515, 545, 575, 595, 615, and 665 nm.

Graphic Jump Location
Fig. 4
F4 :

(a) Gastric ulcer region in conventional color imaging mode. (b) Enhanced image using HS images at 645, 545, and 495 nm as new R, G, and B components, respectively. (c) Enhanced image using HS images at 645, 545, and 465 nm as new R, G, and B components, respectively, (b) and (c) were obtained after white balance processing.

Graphic Jump Location
Fig. 3
F3 :

Spectral curves from representative points in HS image cube. There are three typical regions: ulcer region (gastrohelcosis), erythematous region, and normal tissue region.

Tables

Table Grahic Jump Location
Table 1Band selection results obtained using DI algorithm on HS images of gastric ulcer.

References

Wolfe  W. L., Introduction to Imaging Spectrometers. ,  SPIE Press ,  Bellingham, Washington  (1997).
Lu  G. L., and Fei  B. W., “Medical hyperspectral imaging: a review,” J. Biomed. Opt.. 19, (1 ), 010901  (2014).CrossRef
Wang  L. V., Photoacoustic Imaging and Spectroscopy. ,  CRC Press ,  Boca Raton, Florida  (2009).
Boas  D. A., , Pitris,  C., and Ramanujam  N., Handbook of Biomedical Optics. ,  CRC Press ,  Boca Raton, Florida  (2012).
Tuchin  V. V., and Tuchin  V., Tissue Optics: Light Scattering Methods and Instruments for Medical Diagnosis. ,  SPIE Press ,  Bellingham, Washington  (2007).
Pierce  M. C.  et al., “Accuracy of in vivo multimodal optical imaging for detection of oral neoplasia,” Cancer Prev. Res.. 5, (6 ), 801 –809 (2012).CrossRef
Akbari  H.  et al., “Hyperspectral imaging and quantitative analysis for prostate cancer detection,” J. Biomed. Opt.. 17, (7 ), 076005  (2012).CrossRef
Masood  K.  et al., “Hyperspectral colon tissue classification using morphological analysis,” in  Proc. Second Int. Conf. on Emerging Technologies 2006 , pp. 735 –741 (2006).
Lindsley  E., , Wachman  E. S., and Farkas  D. L., “The hyperspectral imaging endoscope: a new tool for in vivo cancer detection,” Proc. SPIE. 5322, , 75 –82 (2004).CrossRef
Gerstner  A. O. H.  et al., “Hyperspectral imaging of mucosal surfaces in patients,” J. Biophotonics. 5, (3 ), 255 –262 (2012).CrossRef
Martin  M. E.  et al., “Development of an advanced hyperspectral imaging (HSI) system with applications for cancer detection,” Ann. Biomed. Eng.. 34, (6 ), 1061 –1068 (2006).CrossRef
Martin  R., , Thies  B., and Gerstner  A. O. H., “Hyperspectral hybrid method classification for detecting altered mucosa of the human larynx,” Int. J. Health Geographics. 11, , 21  (2012).
Clancy  N. T.  et al., “Multispectral image alignment using a three channel endoscope in vivo during minimally invasive surgery,” Biomed. Opt. Express. 3, (10 ), 2567 –2578 (2012).CrossRef
Leitner  R.  et al., “Multi-spectral video endoscopy system for the detection of cancerous tissue,” Pattern Recognit. Lett.. 34, (1 ), 85 –93 (2013)CrossRef
Kiyotoki  S.  et al., “New method for detection of gastric cancer by hyperspectral imaging: a pilot study,” J. Biomed. Opt.. 18, (2 ), 026010  (2013)CrossRef
Han  Z.  et al., “In vivo use of hyperspectral imaging to develop a noncontact endoscopic diagnosis support system for malignant colorectal tumors,” J. Biomed. Opt.. 21, (1 ), 016001  (2016)CrossRef
Chaudhuri  S.  et al., “Detection of blood-vessels in retinal images using two-dimensional matched-filters,” IEEE Trans. Med. Imaging. 8, (3 ), 263 –269 (1989)CrossRef
Crum  W. R., , Hartkens  T., and Hill  D. L. G., “Non-rigid image registration: theory and practice,” Br. J. Radiol.. 77, , S140 –S153 (2004).CrossRef
Kybic  J., “Elastic image registration using parametric deformation models,” Citeseer. , PhD dissertation, 13 –159 (2001).
Lange  H.  et al., “Reflectance and fluorescence hyperspectral elastic image registration,” Proc. SPIE. 5370, , 11  (2004).CrossRef
Mitra  R. S., and Murthy  N. N., “Elastic, maximal matching,” Pattern Recognit.. 24, (8 ), 747 –753 (1991).CrossRef
Zitova  B., and Flusser  J., “Image registration methods: a survey,” Image Vision Comput.. 21, (11 ), 977 –1000 (2003).CrossRef
Guo  B. F.  et al., “Band, selection for hyperspectral image classification using mutual information,” IEEE Geosci. Remote Sens. Lett.. 3, (4 ), 522 –526 (2006).CrossRef
Sotoca  J. M., , Pla  F., and Klaren  A. C., “Unsupervised band selection for multispectral images using information theory,” in  Proc. of the 17th Int. Conf. on Pattern Recognition , Vol. 3, , pp. 510 –513 (2004).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Topic Collections

PubMed Articles
Advertisement
  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Article
Sign in or Create a personal account to Buy this article ($20 for members, $25 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.