Video-based multimodal personality analysis

Ziqing Li; Zhenyuan Yang; Jiawen Zhang; Lianggangxu Chen; Gaoqi He

doi:10.1117/12.2623646

16 February 2022 Video-based multimodal personality analysis

Ziqing Li, Zhenyuan Yang, Jiawen Zhang, Lianggangxu Chen, Gaoqi He

Proceedings Volume 12083, Thirteenth International Conference on Graphics and Image Processing (ICGIP 2021); 1208324 (2022) https://doi.org/10.1117/12.2623646
Event: Thirteenth International Conference on Graphics and Image Processing (ICGIP 2021), 2021, Kunming, China

Abstract

Personality analysis defines the task of computing human personality based on their behavior. Traditionally psychological personality research experiment depending on the subjective answers of participants has many inherent challenges. In this paper, we propose a multimodal personality analysis model based on audio features, facial features and environmental features to accurately analyze Big Five traits of people personality. In the feature extraction stage, MFCC features are extracted as the representation of audio data, for facial and environmental characteristics, ResNet and LSTM are used to extract features and integrate time information respectively. We then evaluate our model using the ChaLearn First Impressions V2 challenge dataset and finally reached an accuracy of 89.12%. We demonstrate that multimodal method is more effective for personality analysis, comparing with most of the methods using single modal. Finally, a representational personality analysis system is presented based on pyqt5. It has the function of taking photos or uploading videos to portray the character's personality.

Citation Download Citation

Ziqing Li, Zhenyuan Yang, Jiawen Zhang, Lianggangxu Chen, and Gaoqi He "Video-based multimodal personality analysis", Proc. SPIE 12083, Thirteenth International Conference on Graphics and Image Processing (ICGIP 2021), 1208324 (16 February 2022); https://doi.org/10.1117/12.2623646

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
12 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Video

Feature extraction

Facial recognition systems

Analytical research

Visualization

Data modeling

Biological research

Show All Keywords

Keywords/Phrases

Search In:

Publication Years