Analysis of Feature Space for a 2d/3d Vision based Emotion Recognition Method
In modern human computer interaction systems
(HCI), emotion recognition is becoming an imperative characteristic.
The quest for effective and reliable emotion recognition in HCI has
resulted in a need for better face detection, feature extraction and
classification. In this paper we present results of feature space analysis
after briefly explaining our fully automatic vision based emotion
recognition method. We demonstrate the compactness of the feature
space and show how the 2d/3d based method achieves superior features
for the purpose of emotion classification. Also it is exposed that
through feature normalization a widely person independent feature
space is created. As a consequence, the classifier architecture has
only a minor influence on the classification result. This is particularly
elucidated with the help of confusion matrices. For this purpose
advanced classification algorithms, such as Support Vector Machines
and Artificial Neural Networks are employed, as well as the simple k-
Nearest Neighbor classifier.
[1] P.E. Ekman, W.V. Friesen. Facial Action Coding System. Consulting
Psychologists Press, Palo Alto, CA, 1978.
[2] M. Kunz, V. Mylius, K. Schepelmann, S. Lautenbacher: Impact of age
on the facial expression of pain. J of Psychosom Res 2008; 64:311-318.
[3] C.A. Gilbert, C.M. Lilley, K.D. Craig, P.J. McGrath, C.A. Court,
S.M. Bennett, C.J. Montgomery: Postoperative Pain Expression in Preschool
Children: Validation of the Child Facial Coding System. Clin J
Pain 1999; 15:192-200.
[4] G. Littlewort, M.S. Bartlett, I. Fasel, K. Lee: Faces of Pain: Automated
Measurement of Spontaneous Facial Expressions of Genuine and Posed
Pain. In ICMI -07: Proceedings of the 9th international conference on
Multimodal interfaces, pp. 15-21. 2007.
[5] N. Esau, L. Kleinjohann, B. Kleinjohann: Integration of Emotional
Reactions on Human Facial Expressions into the Robot Head MEXI. In
Proceedings of the IEEE/RSJ International Conference on Intelligent
Robots and Systems (IEEE/RSJ IROS 2007). 2007.
[6] C. Hu, R. Feris, M. Turk, Real-time View-based Face Alignment using
Active Wavelet Networks, Proc. IEEE, IEEE Int'l Workshop on Analysis
and Modeling of Faces and Gestures, pp.215, 2003.
[7] D. Vukadinovic, M. Pantic: Fully automatic facial feature point detection
using Gabor feature based boosted classifiers, Systems, Man and
Cybernetics, 2005 IEEE International Conference on, vol 2, pp. 1692-
1698, 2005.
[8] N. Esau, E. Wetzel, L. Kleinjohann, B. Kleinjohann: Real-Time Facial
Expression Recognition Using a Fuzzy Emotion Model. In Proceedings
of the IEEE International Conference on Fuzzy Systems (FUZZ-IEEE
2007). 2007.
[9] A.B. Ashraf, S. Lucey, J. Cohn, T. Chen, Z. Ambadar, K. Prkachin,
P. Solomon, B.J. Eheobald: The Painful Face - Pain Expression Recognition
Using Active Appearance Models: In ICMI. 2007.
[10] M. Monwar, S. Rezaej, K. Prkachin: Eigenimages Based Pain Expression
Recognition. In IAENG International Journal of Applied Mathematics,
36:2. 2007.
[11] S. Brahnam, C.F. Chuang, F.Y. Shih, M.R. Slack: SVM Classification of
Neonatal Facial Images of Pain. Fuzzy Logic and Applications, WILF
2005, Crema, Italy, 15-17, 2005, LNCS, vol. 3849, 2006b.
[12] G. Littlewort, M.S. Bartlett, I. Fasel, J. Susskind, J. Movellan, Dynamics
of Facial Expression Extracted Automatically from Video. Image and
Vision Computing, vol. 24, pp. 615-625, 2006.
[13] Pantic, M., Pentland, A., Nijholt, A., Huang, T.S.: Human Computing
and Machine Understanding of Human Behavior: A Survey, in Artificial
Intelligence for Human Computing, 2007.
[14] P. Hancock, C. Frowd, E. Brodie, C. Niven: Recognition of Pain Expressions.
Progress in Neural Processing, vol. 16, pp. 339-348, 2005.
[15] S.Z. Li., A.K. Jain: Handbook of Face Recognition, ISBN: 0-387-40595-
X, 2005.
[16] P. Viola, M. Jones: Rapid object detection using a boosted cascade of
simple features, Proceedings IEEE Conf. on Computer Vision and Pattern
Recognition, 2001.
[17] B. Lucas, T. Kanade: An Iterative Image Registration Technique with an
Application to Stereo Vision, Proc. of 7th International Joint Conference
on Artificial Intelligence (IJCAI), pp. 674-679, 1981.
[18] J. Albertz, W. Kreiling: Photogrammetric Guide, Herbert Wichmann
Verlag GmbH, Karlsruhe, 1989.
[19] A. Al-Hamadi, R. Niese, B. Michaelis: A robust approach for contour
extraction and tracking of moving objects in video sequences, Signal
processing, pattern recognition, and applications (Greece June 30 - July
2, 2003) proceedings: Acta Press, pp. 336 - 341, 2003.
[20] V. Blanz, T. Vetter: A Morphable Model for the Synthesis of 3D Faces,
SIGGRAPH, Conference Proceedings, 187-194, 1999.
[21] P. Albrecht, B. Michaelis: Stereo Photogrammetry with Improved
Spatial Resolution, ICPR, pp. 845, 1998.
[22] R. Niese, A. Al-Hamadi, B. Michaelis: A Stereo and Color-based Method
for Face Pose Estimation and Facial Feature Extraction. ICPR 2006
(IEEE), The 18th International Conference on Pattern Recognition,
2006, Hong Kong, Volume: 1, pp. 299-302, 2006.
[23] S. Rusinkiewicz, M. Levoy: Efficient variants of the ICP algorithm,
Proc. of the 3rd Int. Conf. on 3D Digital Imaging & Modeling, pp. 145-
152, 2001.
[24] R. Calow, B. Michaelis: Markerless Analysis of Human Gait with a
Multi-Camera-System. Proceedings of IASTED International Conference
Biomedical Engineering (BioMED), pp. 270-275, 2005.
[25] S. Wachter: Verfolgung von Personen, Universität Karlsruhe, Dissertation,
ISBN-13: 978-3980321266, 1997.
[26] J.L. Bentley: Multidimensional binary search trees used for associative
searching, ACM, 18, pp. 509-517, 1975.
[27] N. Cristianini and J.S Taylor, "An Introduction to Support Vector Machines
and other kernel based learning methods", ISBN: 0-521-78019-X,
2001.
[28] R. Herbrich: Learning kernel Classifiers: theory and algorithms,
ISBN:0-262-08306-X, 2003.
[29] T.F. Wu, C.J. Lin: Probability Estimates for Multi-class Classification
by Pair wise Coupling, Journal of Machine Learning Research 5, 975-
1005, 2004.
[30] Chang, C.-C., Lin., C.-J., LIBSVM: a library for support vector machines,
Available at http://www.csie.ntu.edu.tw/~cjlin/libsvm, 2009.
[1] P.E. Ekman, W.V. Friesen. Facial Action Coding System. Consulting
Psychologists Press, Palo Alto, CA, 1978.
[2] M. Kunz, V. Mylius, K. Schepelmann, S. Lautenbacher: Impact of age
on the facial expression of pain. J of Psychosom Res 2008; 64:311-318.
[3] C.A. Gilbert, C.M. Lilley, K.D. Craig, P.J. McGrath, C.A. Court,
S.M. Bennett, C.J. Montgomery: Postoperative Pain Expression in Preschool
Children: Validation of the Child Facial Coding System. Clin J
Pain 1999; 15:192-200.
[4] G. Littlewort, M.S. Bartlett, I. Fasel, K. Lee: Faces of Pain: Automated
Measurement of Spontaneous Facial Expressions of Genuine and Posed
Pain. In ICMI -07: Proceedings of the 9th international conference on
Multimodal interfaces, pp. 15-21. 2007.
[5] N. Esau, L. Kleinjohann, B. Kleinjohann: Integration of Emotional
Reactions on Human Facial Expressions into the Robot Head MEXI. In
Proceedings of the IEEE/RSJ International Conference on Intelligent
Robots and Systems (IEEE/RSJ IROS 2007). 2007.
[6] C. Hu, R. Feris, M. Turk, Real-time View-based Face Alignment using
Active Wavelet Networks, Proc. IEEE, IEEE Int'l Workshop on Analysis
and Modeling of Faces and Gestures, pp.215, 2003.
[7] D. Vukadinovic, M. Pantic: Fully automatic facial feature point detection
using Gabor feature based boosted classifiers, Systems, Man and
Cybernetics, 2005 IEEE International Conference on, vol 2, pp. 1692-
1698, 2005.
[8] N. Esau, E. Wetzel, L. Kleinjohann, B. Kleinjohann: Real-Time Facial
Expression Recognition Using a Fuzzy Emotion Model. In Proceedings
of the IEEE International Conference on Fuzzy Systems (FUZZ-IEEE
2007). 2007.
[9] A.B. Ashraf, S. Lucey, J. Cohn, T. Chen, Z. Ambadar, K. Prkachin,
P. Solomon, B.J. Eheobald: The Painful Face - Pain Expression Recognition
Using Active Appearance Models: In ICMI. 2007.
[10] M. Monwar, S. Rezaej, K. Prkachin: Eigenimages Based Pain Expression
Recognition. In IAENG International Journal of Applied Mathematics,
36:2. 2007.
[11] S. Brahnam, C.F. Chuang, F.Y. Shih, M.R. Slack: SVM Classification of
Neonatal Facial Images of Pain. Fuzzy Logic and Applications, WILF
2005, Crema, Italy, 15-17, 2005, LNCS, vol. 3849, 2006b.
[12] G. Littlewort, M.S. Bartlett, I. Fasel, J. Susskind, J. Movellan, Dynamics
of Facial Expression Extracted Automatically from Video. Image and
Vision Computing, vol. 24, pp. 615-625, 2006.
[13] Pantic, M., Pentland, A., Nijholt, A., Huang, T.S.: Human Computing
and Machine Understanding of Human Behavior: A Survey, in Artificial
Intelligence for Human Computing, 2007.
[14] P. Hancock, C. Frowd, E. Brodie, C. Niven: Recognition of Pain Expressions.
Progress in Neural Processing, vol. 16, pp. 339-348, 2005.
[15] S.Z. Li., A.K. Jain: Handbook of Face Recognition, ISBN: 0-387-40595-
X, 2005.
[16] P. Viola, M. Jones: Rapid object detection using a boosted cascade of
simple features, Proceedings IEEE Conf. on Computer Vision and Pattern
Recognition, 2001.
[17] B. Lucas, T. Kanade: An Iterative Image Registration Technique with an
Application to Stereo Vision, Proc. of 7th International Joint Conference
on Artificial Intelligence (IJCAI), pp. 674-679, 1981.
[18] J. Albertz, W. Kreiling: Photogrammetric Guide, Herbert Wichmann
Verlag GmbH, Karlsruhe, 1989.
[19] A. Al-Hamadi, R. Niese, B. Michaelis: A robust approach for contour
extraction and tracking of moving objects in video sequences, Signal
processing, pattern recognition, and applications (Greece June 30 - July
2, 2003) proceedings: Acta Press, pp. 336 - 341, 2003.
[20] V. Blanz, T. Vetter: A Morphable Model for the Synthesis of 3D Faces,
SIGGRAPH, Conference Proceedings, 187-194, 1999.
[21] P. Albrecht, B. Michaelis: Stereo Photogrammetry with Improved
Spatial Resolution, ICPR, pp. 845, 1998.
[22] R. Niese, A. Al-Hamadi, B. Michaelis: A Stereo and Color-based Method
for Face Pose Estimation and Facial Feature Extraction. ICPR 2006
(IEEE), The 18th International Conference on Pattern Recognition,
2006, Hong Kong, Volume: 1, pp. 299-302, 2006.
[23] S. Rusinkiewicz, M. Levoy: Efficient variants of the ICP algorithm,
Proc. of the 3rd Int. Conf. on 3D Digital Imaging & Modeling, pp. 145-
152, 2001.
[24] R. Calow, B. Michaelis: Markerless Analysis of Human Gait with a
Multi-Camera-System. Proceedings of IASTED International Conference
Biomedical Engineering (BioMED), pp. 270-275, 2005.
[25] S. Wachter: Verfolgung von Personen, Universität Karlsruhe, Dissertation,
ISBN-13: 978-3980321266, 1997.
[26] J.L. Bentley: Multidimensional binary search trees used for associative
searching, ACM, 18, pp. 509-517, 1975.
[27] N. Cristianini and J.S Taylor, "An Introduction to Support Vector Machines
and other kernel based learning methods", ISBN: 0-521-78019-X,
2001.
[28] R. Herbrich: Learning kernel Classifiers: theory and algorithms,
ISBN:0-262-08306-X, 2003.
[29] T.F. Wu, C.J. Lin: Probability Estimates for Multi-class Classification
by Pair wise Coupling, Journal of Machine Learning Research 5, 975-
1005, 2004.
[30] Chang, C.-C., Lin., C.-J., LIBSVM: a library for support vector machines,
Available at http://www.csie.ntu.edu.tw/~cjlin/libsvm, 2009.
@article{"International Journal of Electrical, Electronic and Communication Sciences:53151", author = "Robert Niese and Ayoub Al-Hamadi and Bernd Michaelis", title = "Analysis of Feature Space for a 2d/3d Vision based Emotion Recognition Method", abstract = "In modern human computer interaction systems
(HCI), emotion recognition is becoming an imperative characteristic.
The quest for effective and reliable emotion recognition in HCI has
resulted in a need for better face detection, feature extraction and
classification. In this paper we present results of feature space analysis
after briefly explaining our fully automatic vision based emotion
recognition method. We demonstrate the compactness of the feature
space and show how the 2d/3d based method achieves superior features
for the purpose of emotion classification. Also it is exposed that
through feature normalization a widely person independent feature
space is created. As a consequence, the classifier architecture has
only a minor influence on the classification result. This is particularly
elucidated with the help of confusion matrices. For this purpose
advanced classification algorithms, such as Support Vector Machines
and Artificial Neural Networks are employed, as well as the simple k-
Nearest Neighbor classifier.", keywords = "Facial expression analysis, Feature extraction, Image processing, Pattern Recognition, Application.", volume = "3", number = "11", pages = "1976-6", }
