Structure-preserving video super-resolution using three-dimensional convolutional neural networks

Chenyu Liu; Xueming Li; Xianlin Zhang; Xianlin Zhang

doi:10.1117/1.JEI.28.2.021007

7 February 2019 Structure-preserving video super-resolution using three-dimensional convolutional neural networks

Chenyu Liu, Xueming Li, Xianlin Zhang, Xianlin Zhang

Author Affiliations +

Journal of Electronic Imaging, Vol. 28, Issue 2, 021007 (February 2019). https://doi.org/10.1117/1.JEI.28.2.021007

Abstract

Convolutional neural networks (CNN) have given rise to a new generation of video super-resolution (SR) technique. However, most existing CNN-based video SR algorithms treat the consecutive frames as a series of feature maps, just as the procedure performed in single image SR algorithms. We propose an end-to-end three-dimensional (3-D) CNN video SR framework. The input frames are considered as a cube in our framework. 3-D convolution is performed on it to extract features along spatial and temporal dimension. Image prior knowledge, such as optical flows, is introduced in reconstruction. A combination of mean square error loss and multiscale structure similarity index (MS-SSIM) loss is used to optimize the model. Experimental results show that the proposed method reconstructs high-resolution frames with more accurate and visually pleasant structures compared with state-of-the-art video SR algorithms. We also achieve comparable PSNR/SSIM results with less computation time.

Citation Download Citation

Chenyu Liu, Xueming Li, Xianlin Zhang, and Xianlin Zhang "Structure-preserving video super-resolution using three-dimensional convolutional neural networks," Journal of Electronic Imaging 28(2), 021007 (7 February 2019). https://doi.org/10.1117/1.JEI.28.2.021007

Received: 8 September 2018; Accepted: 9 January 2019; Published: 7 February 2019

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available