Sparse video representation using steered mixture-of-experts with global motion compensation

Rolf Jongebloed; Erik Bochinski; Thomas Sikora

doi:10.1117/12.2665600

7 June 2023 Sparse video representation using steered mixture-of-experts with global motion compensation

Rolf Jongebloed, Erik Bochinski, Thomas Sikora

Proceedings Volume 12571, Real-time Processing of Image, Depth and Video Information 2023; 125710J (2023) https://doi.org/10.1117/12.2665600
Event: SPIE Optics + Optoelectronics, 2023, Prague, Czech Republic

Abstract

Steered-Mixtures-of-Experts (SMoE) present a unified framework for sparse representation and compression of image data with arbitrary dimensionality. Recent work has shown great improvements in the performance of such models for image and light-field representation. However, for the case of videos the straight-forward application yields limited success as the SMoE framework leads to a piece-wise linear representation of the underlying imagery which is disrupted by nonlinear motion. We incorporate a global motion model into the SMoE framework which allows for higher temporal steering of the kernels. This drastically increases its capabilities to exploit correlations between adjacent frames by only adding 2 to 8 motion parameters per frame to the model but decreasing the required amount of kernels on average by 54.25%, respectively, while maintaining the same reconstruction quality yielding higher compression gains. By halving the number of necessary kernels, we achieve a significant reduction in complexity on the decoder side being a crucial step towards real-time processing.

Conference Presentation

Citation Download Citation

Rolf Jongebloed, Erik Bochinski, and Thomas Sikora "Sparse video representation using steered mixture-of-experts with global motion compensation", Proc. SPIE 12571, Real-time Processing of Image, Depth and Video Information 2023, 125710J (7 June 2023); https://doi.org/10.1117/12.2665600

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available