A Hidden Markov Model-Based Isolated and Meaningful Hand Gesture Recognition
Gesture recognition is a challenging task for extracting
meaningful gesture from continuous hand motion. In this paper, we propose an automatic system that recognizes isolated gesture,
in addition meaningful gesture from continuous hand motion for Arabic numbers from 0 to 9 in real-time based on Hidden Markov Models (HMM). In order to handle isolated gesture, HMM using
Ergodic, Left-Right (LR) and Left-Right Banded (LRB) topologies is applied over the discrete vector feature that is extracted from stereo
color image sequences. These topologies are considered to different
number of states ranging from 3 to 10. A new system is developed to recognize the meaningful gesture based on zero-codeword detection
with static velocity motion for continuous gesture. Therefore, the
LRB topology in conjunction with Baum-Welch (BW) algorithm for
training and forward algorithm with Viterbi path for testing presents the best performance. Experimental results show that the proposed system can successfully recognize isolated and meaningful gesture and achieve average rate recognition 98.6% and 94.29% respectively.
[1] X. Deyou, A Network Approach for Hand Gesture Recognition in Virtual
Reality Driving Training System of SPG, In International Conference on
Pattern Recognition (ICPR 06), pp. 519-522, 2006.
[2] D. B. Nguyen, S. Enokida, and E. Toshiaki, Real-Time Hand Tracking and
Gesture Recognition System, IGVIP05 Conference, CICC, pp. 362-368,
2005.
[3] N. P. Vassilia, and G. M. Konstantinos, On Feature Extraction and Sign
Recognition for Greek Sign Language, Proceedings of the 7th IASTED
International Conference Artificial Intelligence and Soft Computer, pp.
93-98, 2003.
[4] M. Elmezain, A. Al-Hamadi, and B. Michaelis, Real-Time Capable System for Hand Gesture Recognition Using Hidden Markov Models in
Stereo Color Image Sequences, The Journal of W S C G-08, Vol. 16(1),
pp. 65-72, 2008.
[5] E. Holden, R. Owens, and G. Roy, Hand Movement Classification Using
Adaptive Fuzzy Expert System, Journal of expert Systems Research and
Application, Vol. 9(4), pp. 465-480, 1996.
[6] L. Nianjun, C. L. Brian, J. K. Peter, and A. D. Richard, Model Structure
Selection & Training Algorithms for a HMM Gesture Recognition System,
In International Workshop in Frontiers of Handwriting Recognition, pp.
100-106, 2004.
[7] H. Lee, and J. Kim, An HMM-Based Threshold Model Approach for Gesture Recognition, IEEE Trans.on Pattern Analysis and Machine Intelligence, Vol. 21(10), pp. 961-973, 1999.
[8] Y. Ming-Hsuan, and A. Narendra, Gaussian Mixture Modeling of Human
Skin Color and Its Applications in Image and Video Databases, In the
SPIE/EI&T Storage and Retrieval for Image and Video Databases, pp.
458-466, 1999.
[9] S. Askar, Y. Kondratyuk, K. Elazouzi, P. Kauff, and O. Schreer, Vision-
Based Skin-Colour Segmentation of Moving Hands for Real-Time Application, 1st European CVMP, pp. 79-85, 2004.
[10] S. L. Phung, A. Bouzerdoum, and D. Chai, A Novel Skin Color Model
in Y CbCr Color Space and its Application to Human Face Detection, In IEEE International Conference on Image Processing (ICIP), pp. 289-292,2002.
[11] Y. Raja, S. J. Mckenna, and S. Gong, Colour Model Selection and
Adaptation in Dynamic Scenes, In Proceedings European Conference on
Computer Vision, pp. 460-474, 1998.
[12] A. R. Richard, and F. W. Homer, Mixture Densities, Maximum Likelihood
and the EM Algorithm, SIAM Review, Vol. 26(2), pp. 195-239, 1984.
[13] R. Niese, A. Al-Hamadi, and B. Michaelis, A Stereo and Color-based
Method for Face Pose Estimation and Facial Feature Extraction, The
IEEE 18th International Conference on Pattern Recognition, Vol. 1, pp.
299- 302, 2006.
[14] R. Niese, A. Al-Hamadi, and B. Michaelis, A Novel Method for 3D Face
Detection and Normalization, Journal of Multimedia, Vol. 2(5), pp. 1-12, 2007.
[15] R. R. Lawrence, A Tutorial on Hidden Markov Models and Selected
Applications in Speech Recognition, Proceeding of the IEEE, Vol. 77(2),
pp. 257-286, 1989.
[16] S. Mitra, and T. Acharya, Gesture Recognition: A Survey, IEEE Transactions
on Systems, MAN, and Cybernetics, pp. 311-324, 2007.
[17] H. Yoon, J. Soh, Y. J. Bae, and H. S. Yang, Hand Gesture Recognition
using Combined Features of Location, Angle and Velocity, Pattern Recognition 34(7), pp. 1491-1501, 2001.
[18] M. Elmezain, A. Al-Hamadi, and B. Michaelis, Gesture Recognition for
Alphabets from Hand Motion Trajectory Using Hidden Markov Models,
The IEEE International Symposium on Signal Processing and Information
Technology, pp. 1209-1214, 2007.
[1] X. Deyou, A Network Approach for Hand Gesture Recognition in Virtual
Reality Driving Training System of SPG, In International Conference on
Pattern Recognition (ICPR 06), pp. 519-522, 2006.
[2] D. B. Nguyen, S. Enokida, and E. Toshiaki, Real-Time Hand Tracking and
Gesture Recognition System, IGVIP05 Conference, CICC, pp. 362-368,
2005.
[3] N. P. Vassilia, and G. M. Konstantinos, On Feature Extraction and Sign
Recognition for Greek Sign Language, Proceedings of the 7th IASTED
International Conference Artificial Intelligence and Soft Computer, pp.
93-98, 2003.
[4] M. Elmezain, A. Al-Hamadi, and B. Michaelis, Real-Time Capable System for Hand Gesture Recognition Using Hidden Markov Models in
Stereo Color Image Sequences, The Journal of W S C G-08, Vol. 16(1),
pp. 65-72, 2008.
[5] E. Holden, R. Owens, and G. Roy, Hand Movement Classification Using
Adaptive Fuzzy Expert System, Journal of expert Systems Research and
Application, Vol. 9(4), pp. 465-480, 1996.
[6] L. Nianjun, C. L. Brian, J. K. Peter, and A. D. Richard, Model Structure
Selection & Training Algorithms for a HMM Gesture Recognition System,
In International Workshop in Frontiers of Handwriting Recognition, pp.
100-106, 2004.
[7] H. Lee, and J. Kim, An HMM-Based Threshold Model Approach for Gesture Recognition, IEEE Trans.on Pattern Analysis and Machine Intelligence, Vol. 21(10), pp. 961-973, 1999.
[8] Y. Ming-Hsuan, and A. Narendra, Gaussian Mixture Modeling of Human
Skin Color and Its Applications in Image and Video Databases, In the
SPIE/EI&T Storage and Retrieval for Image and Video Databases, pp.
458-466, 1999.
[9] S. Askar, Y. Kondratyuk, K. Elazouzi, P. Kauff, and O. Schreer, Vision-
Based Skin-Colour Segmentation of Moving Hands for Real-Time Application, 1st European CVMP, pp. 79-85, 2004.
[10] S. L. Phung, A. Bouzerdoum, and D. Chai, A Novel Skin Color Model
in Y CbCr Color Space and its Application to Human Face Detection, In IEEE International Conference on Image Processing (ICIP), pp. 289-292,2002.
[11] Y. Raja, S. J. Mckenna, and S. Gong, Colour Model Selection and
Adaptation in Dynamic Scenes, In Proceedings European Conference on
Computer Vision, pp. 460-474, 1998.
[12] A. R. Richard, and F. W. Homer, Mixture Densities, Maximum Likelihood
and the EM Algorithm, SIAM Review, Vol. 26(2), pp. 195-239, 1984.
[13] R. Niese, A. Al-Hamadi, and B. Michaelis, A Stereo and Color-based
Method for Face Pose Estimation and Facial Feature Extraction, The
IEEE 18th International Conference on Pattern Recognition, Vol. 1, pp.
299- 302, 2006.
[14] R. Niese, A. Al-Hamadi, and B. Michaelis, A Novel Method for 3D Face
Detection and Normalization, Journal of Multimedia, Vol. 2(5), pp. 1-12, 2007.
[15] R. R. Lawrence, A Tutorial on Hidden Markov Models and Selected
Applications in Speech Recognition, Proceeding of the IEEE, Vol. 77(2),
pp. 257-286, 1989.
[16] S. Mitra, and T. Acharya, Gesture Recognition: A Survey, IEEE Transactions
on Systems, MAN, and Cybernetics, pp. 311-324, 2007.
[17] H. Yoon, J. Soh, Y. J. Bae, and H. S. Yang, Hand Gesture Recognition
using Combined Features of Location, Angle and Velocity, Pattern Recognition 34(7), pp. 1491-1501, 2001.
[18] M. Elmezain, A. Al-Hamadi, and B. Michaelis, Gesture Recognition for
Alphabets from Hand Motion Trajectory Using Hidden Markov Models,
The IEEE International Symposium on Signal Processing and Information
Technology, pp. 1209-1214, 2007.
@article{"International Journal of Electrical, Electronic and Communication Sciences:50637", author = "Mahmoud Elmezain and Ayoub Al-Hamadi and Jörg Appenrodt and Bernd Michaelis", title = "A Hidden Markov Model-Based Isolated and Meaningful Hand Gesture Recognition", abstract = "Gesture recognition is a challenging task for extracting
meaningful gesture from continuous hand motion. In this paper, we propose an automatic system that recognizes isolated gesture,
in addition meaningful gesture from continuous hand motion for Arabic numbers from 0 to 9 in real-time based on Hidden Markov Models (HMM). In order to handle isolated gesture, HMM using
Ergodic, Left-Right (LR) and Left-Right Banded (LRB) topologies is applied over the discrete vector feature that is extracted from stereo
color image sequences. These topologies are considered to different
number of states ranging from 3 to 10. A new system is developed to recognize the meaningful gesture based on zero-codeword detection
with static velocity motion for continuous gesture. Therefore, the
LRB topology in conjunction with Baum-Welch (BW) algorithm for
training and forward algorithm with Viterbi path for testing presents the best performance. Experimental results show that the proposed system can successfully recognize isolated and meaningful gesture and achieve average rate recognition 98.6% and 94.29% respectively.", keywords = "Computer Vision & Image Processing, Gesture Recognition, Pattern Recognition, Application", volume = "2", number = "5", pages = "760-8", }
