A selective fusion module for video super resolution with recurrent architecture

Zichen Gong; Toshiya Hori; Hiroshi Watanabe; Tomohiro Ikai; Takeshi Chujoh; Eiichi Sasaki; Norio Ito

doi:10.1117/12.2566219

1 June 2020 A selective fusion module for video super resolution with recurrent architecture

Zichen Gong, Toshiya Hori, Hiroshi Watanabe, Tomohiro Ikai, Takeshi Chujoh, Eiichi Sasaki, Norio Ito

Proceedings Volume 11515, International Workshop on Advanced Imaging Technology (IWAIT) 2020; 115151I (2020) https://doi.org/10.1117/12.2566219
Event: International Workshop on Advanced Imaging Technologies 2020 (IWAIT 2020), 2020, Yogyakarta, Indonesia

Abstract

As an important subtask of video restoration, video super-resolution has attracted a lot of attention in the community as it can eventually promote a wide range of technologies, e.g., video transmission system. Recent video super resolution model¹ achieves cutting-edge performance. It efficiently utilizes recurrent architecture with neural networks to gradually aggregate details from previous frames. Nevertheless, this method faces a serious drawback that it is sensitive to occlusion, blur, and large motion changes since it only takes the previous generated output as recurrent input for the super resolution model. This will lead to undesirable rapid information loss during the recurrently generating process, and performance will therefore be dramatically decreased. Our works focus on addressing the issue of rapid information loss in video super resolution model with recurrent architecture. By producing attention maps through selective fusion module, the recurrent model can adaptively aggregate necessary details across all previously generated high-resolution (HR) frames according to their informativeness. The proposed method is useful for preserving high frequency details collected progressively from each frame while being capable of removing noisy artifacts. This significantly improves the average quality of the super resolution video.

Citation Download Citation

Zichen Gong, Toshiya Hori, Hiroshi Watanabe, Tomohiro Ikai, Takeshi Chujoh, Eiichi Sasaki, and Norio Ito "A selective fusion module for video super resolution with recurrent architecture", Proc. SPIE 11515, International Workshop on Advanced Imaging Technology (IWAIT) 2020, 115151I (1 June 2020); https://doi.org/10.1117/12.2566219

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available