Infrared and passive millimeter wave image fusion based on multi-resolution deep learning method

Zhijia Yang; Aoqi Ma; Zefeng Zhang; Zhaocen Zhang; Zhengjun Li; Kun Gao

doi:10.1117/12.2627192

27 March 2022 Infrared and passive millimeter wave image fusion based on multi-resolution deep learning method

Zhijia Yang, Aoqi Ma, Zefeng Zhang, Zhaocen Zhang, Zhengjun Li, Kun Gao

Proceedings Volume 12169, Eighth Symposium on Novel Photoelectronic Detection Technology and Applications; 12169C8 (2022) https://doi.org/10.1117/12.2627192
Event: Eighth Symposium on Novel Photoelectronic Detection Technology and Applications, 2021, Kunming, China

Abstract

Infrared & Passive Millimeter Wave (IR/PMMW) composite guidance is the development hotspot of multimode composite guidance technology. Considering the low penetrability of IR imaging system under nonideal visibility conditions, while the PMMW imaging technology has high atmospheric transmittance but low resolution, a fusion method of infrared/millimeter wave images based on multi-resolution deep learning is proposed. In this method, the infrared and millimeter wave images are first decomposed by NSCT transformation to separate the low-frequency and high-frequency components of the input image. For low-frequency components fusion, we design a specific generative adversarial convolution neural network for activity-level measurement and fusion rules to preserve information in the both scenes as much as possible. The high-frequency components are fused by the Pulse Couple Neural Network algorithm because of its similar processing mechanism with human visual nervous system; the fusion results of the low and high frequency components are subjected to inverse NSCT transformation, the final fused image is obtained. Data augment technology, such as image style transferring, is applied to extend IR/PMMW training set. Extensive results demonstrate that the proposed method can generate image with higher qualities with salient targets inside, deliver better performance than the state-of-the-art methods in both subjective and objective evaluation.

Citation Download Citation

Zhijia Yang, Aoqi Ma, Zefeng Zhang, Zhaocen Zhang, Zhengjun Li, and Kun Gao "Infrared and passive millimeter wave image fusion based on multi-resolution deep learning method", Proc. SPIE 12169, Eighth Symposium on Novel Photoelectronic Detection Technology and Applications, 12169C8 (27 March 2022); https://doi.org/10.1117/12.2627192

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
6 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Image fusion

Infrared imaging

Infrared radiation

Extremely high frequency

Neural networks

Image processing

Thermography

Show All Keywords

Keywords/Phrases

Search In:

Publication Years