Combining Skin Color and Optical Flow for Computer Vision Systems

Skin color is an important visual cue for computer vision systems involving human users. In this paper we combine skin color and optical flow for detection and tracking of skin regions. We apply these techniques to gesture recognition with encouraging results. We propose a novel skin similarity measure. For grouping detected skin regions we propose a novel skin region grouping mechanism. The proposed techniques work with any number of skin regions making them suitable for a multiuser scenario.

Authors:

• Muhammad Raza Ali
• Tim Morris

Keywords:

• Bayesian tracking
• chromaticity space
• optical flowgesture recognition

References:

[1] M.R. Ali and T. Morris. "Skin Locus Based Skin Detection for
Gesture Recognition". BMVC Postgraduate Workshop, Aberystwyth, 3-
Sep-2010.
[2] A. Argyros and M. Lourakis. "Real time Tracking of Multiple Skin-
Colored Objects with a Possibly Moving Camera". In Proceedings of the
European Conference on Computer Vision (ECCV-04), Springer-
Verlag, vol. 3, pp. 368-379, Prague, Czech Republic, May 11-14, 2004.
[3] A. Del. Bimbo, L. Landucci , and A. Valli. "Multi-User Natural
Interaction System based on Real-Time Hand Tracking and Gesture
Recognition". In Proceedings of ICPR (3). 2006, pp: 55-58.
[4] M. Black and A.D. Jepson. "A probabilistic framework for matching
temporal trajectories: Condensation-based recognition of gestures and
expressions". In Proceedings of Fifth European Conference on
Computer Vision Vol. 1, 1998, pp: 909-924.
[5] T. Coogan, G. Awad, J. Han, and A. Sutherland. "Real time hand
gesture recognition including hand segmentation and tracking". In
Proceedings of ISVC06, 2006 pp: 495 - 504.
[6] P. Dreuw, T. Deselaers, D. Keysers, D, and H. Ney. "Modelling
image variability in appearance-based gesture recognition". In ECCV
Workshop on Statistical Methods in Multi-Image and Video Processing
(ECCV-SMVP), Graz, Austria, May 2006, pp: 7-18.
[7] A. Gelb. "Applied Optimal Estimation". MIT Press, 1996
[8] M.S. Grewal and A.P. Andrews "Kalman Filtering". Prentice Hall, 1993.
[9] C. Hsu, C. Chang, and C.J Lin. "A Practical Guide to Support Vector
Classification". http://www.csie.ntu.edu.tw/~cjlin/
[10] M. Isard and A. Blake. "CONDENSATION -- Conditional Density
Propagation For Visual Tracking". Int. Journal of Computer Vision, Vol:
29(1), 1998, pp: 5-28.
[11] E. Koller-Meier and F. Ade, "Tracking Multiple Objects Using the
Condensation algorithm". Robotics and Autonomous Systems, vol. 34,
2001, pp. 93-105.
[12] B. Lucas and T. Kanade. "An Iterative Image Registration Technique
With An Application To Stereo Vision". Proceedings of Imaging
understanding workshop, 1981, pp: 121-130.
[13] L. Saigol , S. Sclaroff , and V. Athitsos. "Skin Color-Based Video
Segmentation Under Time-Varying Illumination". IEEE Transactions
On Pattern Analysis And Machine Intelligence, Vol. 26, No. 7, July
2004.
[14] J. Santner , M. Werlberger, T. Mauthner, W. Paier , and Bischof H.
"Flow Games". 1st Int. Workshop on Computer Vision for Computer
Games (CVCG) in conjunction with IEEE CVPR, 2010.
[15] G. Shin. and J. Chun, "Vision-based Multimodal Human Computer
Interface based on Parallel Tracking of Eye and Hand Motion". In
Proceedings of International Conference on Convergence Information
Technology , 2007, pp. 2443-2448.
[16] M. Soriano, B. Martinkauppi, S. Huovinen, and M. Laaksonen,
"Adaptive Skin Color Modelling Using The Skin Locus For Selecting
Training Pixels". Pattern Recognition,36(3), 2003, pp: 681-690.
[17] M. Storring , J.H. Andersen, and E. Granum, "Skin Color Detection
Under Changing Lighting Conditions". In Proceedings of Seventh
Symposium Intelligent Robotics Systems,1999. pp. 187-195.
[18] D. Sun, S. Roth and M. Black, Secrets of Optical Flow Estimation and
Their Principles". In proceedings of CVPR, 2010.
http://www.cs.brown.edu/~dqsun/pubs/cvpr_2010_flow.pdf
[19] N. Vo, Q. Tran, T.B. Dinh and Q.M. Nguyen, "An Efficient Human
Computer Interaction Framework Using Skin Color Tracking and
Gesture Recognition". International Conference on Computing and
Communication Technologies, Research, Innovation, and Vision for the
Future (RIVF), 2010.
[20] Y. Wu and T. Huang. "Vision Based Gesture Recognition: A review",
In International Gesture Workshop (GW99),Gif-sur-Yvette, France,
1999.
[21] Y. Yin and D.R. Toward, "Natural Interaction In The Real World:
Real-Time Gesture Recognition". International Conference on
Multimodal Interfaces and the Workshop on Machine Learning for
Multimodal Interaction (ICMI-MLMI '10), Beijing 2010.
[22] S. Zhan, Y. Hanxuan, Z. Yanguo, and Z. Feng, "Hand detection and
gesture recognition exploit motion times image in complicate
scenarios". In Proceedings of the 6th international conference on
Advances in visual computing - Volum. Part II. Springer-Verlag, Berlin,
Heidelberg, 2010, 628-636.