An interactive VR system based on full-body tracking and gesture recognition

Xia Zeng; Xinzhu Sang; Duo Chen; Peng Wang; Nan Guo; Binbin Yan; Kuiru Wang

doi:10.1117/12.2247808

31 October 2016 An interactive VR system based on full-body tracking and gesture recognition

Xia Zeng, Xinzhu Sang, Duo Chen, Peng Wang, Nan Guo, Binbin Yan, Kuiru Wang

Proceedings Volume 10021, Optical Design and Testing VII; 1002129 (2016) https://doi.org/10.1117/12.2247808
Event: SPIE/COS Photonics Asia, 2016, Beijing, China

Abstract

Most current virtual reality (VR) interactions are realized with the hand-held input device which leads to a low degree of presence. There is other solutions using sensors like Leap Motion to recognize the gestures of users in order to interact in a more natural way, but the navigation in these systems is still a problem, because they fail to map the actual walking to virtual walking only with a partial body of the user represented in the synthetic environment. Therefore, we propose a system in which users can walk around in the virtual environment as a humanoid model, selecting menu items and manipulating with the virtual objects using natural hand gestures. With a Kinect depth camera, the system tracks the joints of the user, mapping them to a full virtual body which follows the move of the tracked user. The movements of the feet can be detected to determine whether the user is in walking state, so that the walking of model in the virtual world can be activated and stopped by means of animation control in Unity engine. This method frees the hands of users comparing to traditional navigation way using hand-held device. We use the point cloud data getting from Kinect depth camera to recognize the gestures of users, such as swiping, pressing and manipulating virtual objects. Combining the full body tracking and gestures recognition using Kinect, we achieve our interactive VR system in Unity engine with a high degree of presence.

Citation Download Citation

Xia Zeng, Xinzhu Sang, Duo Chen, Peng Wang, Nan Guo, Binbin Yan, and Kuiru Wang "An interactive VR system based on full-body tracking and gesture recognition", Proc. SPIE 10021, Optical Design and Testing VII, 1002129 (31 October 2016); https://doi.org/10.1117/12.2247808

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
7 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Virtual reality

Gesture recognition

Human-machine interfaces

Navigation systems

Cameras

Sensors

Head-mounted displays

Show All Keywords

Keywords/Phrases

Search In:

Publication Years