Face Localization Using Illumination-dependent Face Model for Visual Speech Recognition

A robust still image face localization algorithm capable of operating in an unconstrained visual environment is proposed. First, construction of a robust skin classifier within a shifted HSV color space is described. Then various filtering operations are performed to better isolate face candidates and mitigate the effect of substantial non-skin regions. Finally, a novel Bhattacharyya-based face detection algorithm is used to compare candidate regions of interest with a unique illumination-dependent face model probability distribution function approximation. Experimental results show a 90% face detection success rate despite the demands of the visually noisy environment.

Authors:

• Robert E. Hursig
• Jane X. Zhang

Keywords:

• Audio-visual speech recognition
• Bhattacharyyacoefficient
• face detection

References:

[1] G. Potamianos, J. Luettin, and I. Matthews, "Audio-Visual Automatic
Speech Recognition: An Oveview," Issues in Visual and Audio-Visual
Speech Processing, MIT Press, rCh 10, 2004.
[2] D.G. Stork and M.E. Hennecke, "Speechreading by Humans and
Machines" in NATO ASI Series F, vol 150, Springer Verlag, 1996.
[3] B. Lee, M. Hasegawa-Johnson, C. Goudeseune, S. Kamdar, S. Borys, M.
Liu, T. Huang, "AVICAR: Audio-Visual Speech Corpus in a Car
Environment," INTERSPEECH2004-ICSLP, 2004.
[4] Viola, Jones, "Robust Real-time Object Detection," IJCV 2001.
[5] P. Delmas, M. Lievin, "From Face Features Analysis to Automatic Lip
Reading. Seventh International Conference on Control, Automation,
Robotics and Vision, vol. 3, Dec. 2-5, 2002.
[6] K. Kumar, C. Tsuhan, R.M. Stern, "Profile View Lip Reading," IEEE
International Conference on Acoustics, Speech and Signal Processing,
vol. 4, April 15-20, 2007.
[7] L.G. da Silveira, J. Facon, D.L. Borges, "Visual Speech Recognition: A
Solution from Feature Extraction to Words Classification," sibgrapi, XVI
Brazilian Symposium on Computer Graphics and Image Processing,
2003.
[8] M.C. Shin, K.I. Chang, L.V. Tsap, "Does Color Space Transformation
Make Any Difference on Skin Detection?" WACV: Proceedings of the
Sixth IEEE Workshop on Applications of Computer Vision, Washington
DC, IEEE Computer Society, 2002.
[9] M.J. Jones, J.M. Rehg, "Statistical Color Models with Application to
Skin Detection," International Journal of Computer Vision, vol. 46,
no.1, 2006
[10] Y. Ming-Hsuan, A. Narendra, "Detecting Human Faces in Color
Images," Proceedings of the International Conference on Image
Processing, vol. 1, 1998.
[11] X. Zhang, H. A. Montoya, and B. Crow, "Finding Lips in Unconstrained
Imagery for Improved Automatic Speech Recognition," Proceedings of
9th International Conference on Visual Information Systems, 2007.
[12] M. Abdel-Mottaleb, A. Ellgammal, "Face Detection in Complex
Environments from Color Images," Proceedings of the International
Conference on Image Processing, vol.3, 1999
[13] B. Crow, "Automated Location and Tracking of Facial Features in an
Unconstrained Environment," Master-s Thesis, California Polytechnic
State University, 2008.
[14] J. van de Weijer, Th. Gevers, and A. Gijsenij, "Edge-based color
constancy," in Trans. On Image Processing, 2007.
[15] G.D. Finlayson and E. Trezzi, , "Shades of gray and colour constancy,"
in Proc. Of the 12th Color Imaging Conference, 2004.