On-line Recognition of Isolated Gestures of Flight Deck Officers (FDO)
The paper presents an on-line recognition machine
(RM) for continuous/isolated, dynamic and static gestures that arise
in Flight Deck Officer (FDO) training. RM is based on generic pattern
recognition framework. Gestures are represented as templates using
summary statistics. The proposed recognition algorithm exploits temporal
and spatial characteristics of gestures via dynamic programming
and Markovian process. The algorithm predicts corresponding index
of incremental input data in the templates in an on-line mode.
Accumulated consistency in the sequence of prediction provides a
similarity measurement (Score) between input data and the templates.
The algorithm provides an intuitive mechanism for automatic detection
of start/end frames of continuous gestures. In the present paper,
we consider isolated gestures. The performance of RM is evaluated
using four datasets - artificial (W TTest), hand motion (Yang) and
FDO (tracker, vision-based ). RM achieves comparable results which
are in agreement with other on-line and off-line algorithms such as
hidden Markov model (HMM) and dynamic time warping (DTW).
The proposed algorithm has the additional advantage of providing
timely feedback for training purposes.
[1] Christopher M. Bishop. Neural networks for pattern recognition. Oxford
University Press, 1996.
[2] Herve Bourlardy and Samy Bengio. The Handbook of Brain Theory
and Neural Networks, chapter Hidden Markov Models and other Finite
State Automata for Sequence Processing. The MIT Press, second edition,
2002.
[3] E. Keogh C. A. Ratanamahatana. Everything you know about dynamic
time warping is wrong. Third Workshop on Mining Temporal and
Sequential Data, in conjunction with the Tenth ACM SIGKDD International
Conference on Knowledge Discovery and Data Mining, 2004.
[4] Andrea Corradini. Dynamic time warping for off-line recognition of a
small gesture vocabulary. In Proceedings of the IEEE ICCV Workshop
on Recognition, Analysis, and Tracking of Faces and Gestures in Real-
Time Systems (RATFG-RTS-01), page 82. IEEE Computer Society, 2001.
[5] S. Sidney Fels and Geoffrey E. Hinton. Glove-Talk: A neural network
interface between a data-glove and a speech synthesizer. IEEE Transactions
on Neural Networks, 4(1):2-8, January 1993.
[6] Yang-Hee Nam Jane Koh. Full-body motion recognition using principle
component based target reduction. In KIPS(Korean Information Processing
Society) Proceedings, volume Vol. 11 , no.1, pages 873-876,
Korea, May 2004.
[7] Yang-Hee Nam Jane Koh, Eun-Woo Lee. Full-body motion recognition
using multi-phase target reduction method. HCI 2004(Korean), 2004.
[8] Yangsheng Xu Jie Yang. Hidden markov model for gesture recognition.
Technical Report CMU-RI-TR-94-10, The Robotics Institute, Carneige
Melon University, 1994.
[9] M. W. Kadous. Temporal Classification: Extending the Classification
Paradigm to Multivariate Time Series. PhD thesis, The University of
New South Wales, School of Computer Science and Engineering, 2002.
[10] C. Lee and Y. Xu. Online, interactive learning of gestures for human/
robot interfaces, 1996.
[11] H. Li and M. Greenspan. Continuous time-varying gesture segmentation
by dynamic time warping of compound gesture models. 2005.
[12] Kouichi Murakami and Hitomi Taguchi. Gesture recognition using
recurrent neural networks. In CHI -91: Proceedings of the SIGCHI
conference on Human factors in computing systems, pages 237-242,
New York, NY, USA, 1991. ACM Press.
[13] Y. Nam and K. Wohn. Recognition of space-time handgestures using
hidden markov model, 1996.
[14] Vladimir Pavlovic, Rajeev Sharma, and Thomas S. Huang. Visual
interpretation of hand gestures for human-computer interaction: A review.
IEEE Transactions on Pattern Analysis and Machine Intelligence,
19(7):677-695, 1997.
[15] Rabiner L. R. A tutorial on hidden markov models and selected
applications in speech recognition. Proc. IEEE, 77, (2):257-286, Feb
1989.
[16] Gerhard Rigoll, Andreas Kosmala, and Stefan Eickeler. High performance
real-time gesture recognition using hidden markov models. In
Proceedings of the International Gesture Workshop on Gesture and Sign
Language in Human-Computer Interaction, pages 69-80, London, UK,
1998. Springer-Verlag.
[17] D T Sodiri and V V S S Sastry. On the interpretation of gestures
arising in flight deck officers training. In Proceedings of the Thirteenth
Conference on Behaviour Representation in Modelling and Simulation,
2004.
[18] Thomas Hain-Phil Woodland Steve Young, Gunnar Evermann. The HTK
Book, 3.2.1. Cambridge Research Laboratory Ltd, 2002.
[19] M Turk. Handbook of virtual environments: Design, implementation,
and applications, chapter Gesture recognition, pages 223-238. Mahwah,
NJ: Lawrence Erlbaum Associates, Inc., 2002.
[20] P. Vamplew and A. Adams. Recognition and anticipation of hand
motions using a recurrent neural network, 1995.
[21] Simei G. Wysoski, Marcus V. Lamar, Susumu Kuroyanagi, and Akira
Iwata. A rotation invariant approach on static-gesture recognition using
boundary histograms and neural networks.
[22] Kiyoung Yang and Cyrus Shahabi. A pca-based similarity measure for
multivariate time series. In Proceedings of the 2nd ACM international
workshop on Multimedia databases, pages 65-74. ACM Press, 2004.
[1] Christopher M. Bishop. Neural networks for pattern recognition. Oxford
University Press, 1996.
[2] Herve Bourlardy and Samy Bengio. The Handbook of Brain Theory
and Neural Networks, chapter Hidden Markov Models and other Finite
State Automata for Sequence Processing. The MIT Press, second edition,
2002.
[3] E. Keogh C. A. Ratanamahatana. Everything you know about dynamic
time warping is wrong. Third Workshop on Mining Temporal and
Sequential Data, in conjunction with the Tenth ACM SIGKDD International
Conference on Knowledge Discovery and Data Mining, 2004.
[4] Andrea Corradini. Dynamic time warping for off-line recognition of a
small gesture vocabulary. In Proceedings of the IEEE ICCV Workshop
on Recognition, Analysis, and Tracking of Faces and Gestures in Real-
Time Systems (RATFG-RTS-01), page 82. IEEE Computer Society, 2001.
[5] S. Sidney Fels and Geoffrey E. Hinton. Glove-Talk: A neural network
interface between a data-glove and a speech synthesizer. IEEE Transactions
on Neural Networks, 4(1):2-8, January 1993.
[6] Yang-Hee Nam Jane Koh. Full-body motion recognition using principle
component based target reduction. In KIPS(Korean Information Processing
Society) Proceedings, volume Vol. 11 , no.1, pages 873-876,
Korea, May 2004.
[7] Yang-Hee Nam Jane Koh, Eun-Woo Lee. Full-body motion recognition
using multi-phase target reduction method. HCI 2004(Korean), 2004.
[8] Yangsheng Xu Jie Yang. Hidden markov model for gesture recognition.
Technical Report CMU-RI-TR-94-10, The Robotics Institute, Carneige
Melon University, 1994.
[9] M. W. Kadous. Temporal Classification: Extending the Classification
Paradigm to Multivariate Time Series. PhD thesis, The University of
New South Wales, School of Computer Science and Engineering, 2002.
[10] C. Lee and Y. Xu. Online, interactive learning of gestures for human/
robot interfaces, 1996.
[11] H. Li and M. Greenspan. Continuous time-varying gesture segmentation
by dynamic time warping of compound gesture models. 2005.
[12] Kouichi Murakami and Hitomi Taguchi. Gesture recognition using
recurrent neural networks. In CHI -91: Proceedings of the SIGCHI
conference on Human factors in computing systems, pages 237-242,
New York, NY, USA, 1991. ACM Press.
[13] Y. Nam and K. Wohn. Recognition of space-time handgestures using
hidden markov model, 1996.
[14] Vladimir Pavlovic, Rajeev Sharma, and Thomas S. Huang. Visual
interpretation of hand gestures for human-computer interaction: A review.
IEEE Transactions on Pattern Analysis and Machine Intelligence,
19(7):677-695, 1997.
[15] Rabiner L. R. A tutorial on hidden markov models and selected
applications in speech recognition. Proc. IEEE, 77, (2):257-286, Feb
1989.
[16] Gerhard Rigoll, Andreas Kosmala, and Stefan Eickeler. High performance
real-time gesture recognition using hidden markov models. In
Proceedings of the International Gesture Workshop on Gesture and Sign
Language in Human-Computer Interaction, pages 69-80, London, UK,
1998. Springer-Verlag.
[17] D T Sodiri and V V S S Sastry. On the interpretation of gestures
arising in flight deck officers training. In Proceedings of the Thirteenth
Conference on Behaviour Representation in Modelling and Simulation,
2004.
[18] Thomas Hain-Phil Woodland Steve Young, Gunnar Evermann. The HTK
Book, 3.2.1. Cambridge Research Laboratory Ltd, 2002.
[19] M Turk. Handbook of virtual environments: Design, implementation,
and applications, chapter Gesture recognition, pages 223-238. Mahwah,
NJ: Lawrence Erlbaum Associates, Inc., 2002.
[20] P. Vamplew and A. Adams. Recognition and anticipation of hand
motions using a recurrent neural network, 1995.
[21] Simei G. Wysoski, Marcus V. Lamar, Susumu Kuroyanagi, and Akira
Iwata. A rotation invariant approach on static-gesture recognition using
boundary histograms and neural networks.
[22] Kiyoung Yang and Cyrus Shahabi. A pca-based similarity measure for
multivariate time series. In Proceedings of the 2nd ACM international
workshop on Multimedia databases, pages 65-74. ACM Press, 2004.
@article{"International Journal of Information, Control and Computer Sciences:64685", author = "Deniz T. Sodiri and Venkat V S S Sastry", title = "On-line Recognition of Isolated Gestures of Flight Deck Officers (FDO)", abstract = "The paper presents an on-line recognition machine
(RM) for continuous/isolated, dynamic and static gestures that arise
in Flight Deck Officer (FDO) training. RM is based on generic pattern
recognition framework. Gestures are represented as templates using
summary statistics. The proposed recognition algorithm exploits temporal
and spatial characteristics of gestures via dynamic programming
and Markovian process. The algorithm predicts corresponding index
of incremental input data in the templates in an on-line mode.
Accumulated consistency in the sequence of prediction provides a
similarity measurement (Score) between input data and the templates.
The algorithm provides an intuitive mechanism for automatic detection
of start/end frames of continuous gestures. In the present paper,
we consider isolated gestures. The performance of RM is evaluated
using four datasets - artificial (W TTest), hand motion (Yang) and
FDO (tracker, vision-based ). RM achieves comparable results which
are in agreement with other on-line and off-line algorithms such as
hidden Markov model (HMM) and dynamic time warping (DTW).
The proposed algorithm has the additional advantage of providing
timely feedback for training purposes.", keywords = "On-line Recognition Algorithm, IsolatedDynamic/Static Gesture Recognition, On-line Markovian/DynamicProgramming, Training in Virtual Environments.", volume = "2", number = "1", pages = "241-7", }
