Lip-reading enhancement for law enforcement

Barry J. Theobald; Richard Harvey; Stephen J. Cox; Colin Lewis; Gari P. Owen

doi:10.1117/12.689960

28 September 2006 Lip-reading enhancement for law enforcement

Barry J. Theobald, Richard Harvey, Stephen J. Cox, Colin Lewis, Gari P. Owen

Proceedings Volume 6402, Optics and Photonics for Counterterrorism and Crime Fighting II; 640205 (2006) https://doi.org/10.1117/12.689960
Event: Optics/Photonics in Security and Defence, 2006, Stockholm, Sweden

Abstract

Accurate lip-reading techniques would be of enormous benefit for agencies involved in counter-terrorism and other law-enforcement areas. Unfortunately, there are very few skilled lip-readers, and it is apparently a difficult skill to transmit, so the area is under-resourced. In this paper we investigate the possibility of making the lip-reading task more amenable to a wider range of operators by enhancing lip movements in video sequences using active appearance models. These are generative, parametric models commonly used to track faces in images and video sequences. The parametric nature of the model allows a face in an image to be encoded in terms of a few tens of parameters, while the generative nature allows faces to be re-synthesised using the parameters. The aim of this study is to determine if exaggerating lip-motions in video sequences by amplifying the parameters of the model improves lip-reading ability. We also present results of lip-reading tests undertaken by experienced (but non-expert) adult subjects who claim to use lip-reading in their speech recognition process. The results, which are comparisons of word error-rates on unprocessed and processed video, are mixed. We find that there appears to be the potential to improve the word error rate but, for the method to improve the intelligibility there is need for more sophisticated tracking and visual modelling. Our technique can also act as an expression or visual gesture amplifier and so has applications to animation and the presentation of information via avatars or synthetic humans.

Citation Download Citation

Barry J. Theobald, Richard Harvey, Stephen J. Cox, Colin Lewis, and Gari P. Owen "Lip-reading enhancement for law enforcement", Proc. SPIE 6402, Optics and Photonics for Counterterrorism and Crime Fighting II, 640205 (28 September 2006); https://doi.org/10.1117/12.689960

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available