Dual-codebook learning and hierarchical transfer for cross-view action recognition

Chengkun Zhang; Huicheng Zheng; Jianhuang Lai

doi:10.1117/1.JEI.27.4.043044

11 August 2018 Dual-codebook learning and hierarchical transfer for cross-view action recognition

Chengkun Zhang, Huicheng Zheng, Jianhuang Lai

Author Affiliations +

Journal of Electronic Imaging, Vol. 27, Issue 4, 043044 (August 2018). https://doi.org/10.1117/1.JEI.27.4.043044

Abstract

We focus on the challenging cross-view action recognition problem. The key to this problem is to find the correspondence between two different views, which is realized in two stages. First, we construct a dual-codebook for the two views, which contains one codebook for each view. Each codeword in one codebook has a corresponding codeword in the other codebook, whereas traditional methods implement independent codebooks for the views. We propose an effective coclustering algorithm based on seminonnegative matrix factorization to derive the dual-codebook. Additionally, to represent actions in one view, unlike most other methods using the codebook of that view only, we also exploit the codebook-specific information from the other view. Thus, we construct mapped-codebooks via codebook transformation, complementing the codebook-to-codebook correspondence within the dual-codebook. In the second stage, observing that the temporal relationship between action segments within an action is view invariant, we further propose a hierarchical transfer framework based on a temporal structure that can effectively capture such action-segment temporal relationship at multiple timescales, which is more discriminative than the usual video-level transfer strategy. Extensive experimental results on the INRIA xmas motion acquisition sequences and West Virginia University datasets demonstrate superiority of the proposed method compared with state-of-the-art approaches.

Citation Download Citation

Chengkun Zhang, Huicheng Zheng, and Jianhuang Lai "Dual-codebook learning and hierarchical transfer for cross-view action recognition," Journal of Electronic Imaging 27(4), 043044 (11 August 2018). https://doi.org/10.1117/1.JEI.27.4.043044

Received: 16 February 2018; Accepted: 20 July 2018; Published: 11 August 2018

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available