Adaptive metric learning with deep neural networks for video-based facial expression recognition

Xiaofeng Liu; Yubin Ge; Chao Yang; Ping Jia

doi:10.1117/1.JEI.27.1.013022

19 February 2018 Adaptive metric learning with deep neural networks for video-based facial expression recognition

Xiaofeng Liu, Yubin Ge, Chao Yang, Ping Jia

Author Affiliations +

Journal of Electronic Imaging, Vol. 27, Issue 1, 013022 (February 2018). https://doi.org/10.1117/1.JEI.27.1.013022

Abstract

Video-based facial expression recognition has become increasingly important for plenty of applications in the real world. Despite that numerous efforts have been made for the single sequence, how to balance the complex distribution of intra- and interclass variations well between sequences has remained a great difficulty in this area. We propose the adaptive ( N+M)-tuplet clusters loss function and optimize it with the softmax loss simultaneously in the training phrase. The variations introduced by personal attributes are alleviated using the similarity measurements of multiple samples in the feature space with many fewer comparison times as conventional deep metric learning approaches, which enables the metric calculations for large data applications (e.g., videos). Both the spatial and temporal relations are well explored by a unified framework that consists of an Inception-ResNet network with long short term memory and the two fully connected layer branches structure. Our proposed method has been evaluated with three well-known databases, and the experimental results show that our method outperforms many state-of-the-art approaches.

Citation Download Citation

Xiaofeng Liu, Yubin Ge, Chao Yang, and Ping Jia "Adaptive metric learning with deep neural networks for video-based facial expression recognition," Journal of Electronic Imaging 27(1), 013022 (19 February 2018). https://doi.org/10.1117/1.JEI.27.1.013022

Received: 8 August 2017; Accepted: 23 January 2018; Published: 19 February 2018

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available