Fusion Classifier for Open-Set Face Recognition with Pose Variations

A fusion classifier composed of two modules, one made by a hidden Markov model (HMM) and the other by a support vector machine (SVM), is proposed to recognize faces with pose variations in open-set recognition settings. The HMM module captures the evolution of facial features across a subject-s face using the subject-s facial images only, without referencing to the faces of others. Because of the captured evolutionary process of facial features, the HMM module retains certain robustness against pose variations, yielding low false rejection rates (FRR) for recognizing faces across poses. This is, however, on the price of poor false acceptance rates (FAR) when recognizing other faces because it is built upon withinclass samples only. The SVM module in the proposed model is developed following a special design able to substantially diminish the FAR and further lower down the FRR. The proposed fusion classifier has been evaluated in performance using the CMU PIE database, and proven effective for open-set face recognition with pose variations. Experiments have also shown that it outperforms the face classifier made by HMM or SVM alone.

• Gee-Sern Jison Hsu

• Face recognition
• open-set identification
• hidden Markov model
• support vector machines.

[1] P.J. Phillips, P. Grother, R.J. Micheals, D.M. Blackburn, E. Tabassi, M.
Bone, "Face Recognition Vendor Test 2002: Evaluation Report," available
at http://www.frvt.org.
[2] V. Blanza and T. Vetter, "Face recognition based on fitting a 3D morphable
model," IEEE Trans. Pattern Analysis and Machine Intelligence
(PAMI), vol. 25, no. 9, pp. 1063-1074, 2003.
[3] H.C. Choi; S.Y Kim; S.H. Oh; S.Y. Oh; S.Y. Cho; "Pose invariant face
recognition with 3D morphable model and neural network," Proc. IEEE
Int-l J. Conf. Neural Networks (IJCNN), 2008, pp. 4131 - 4136.
[4] Q. Chen, J. Yao, W.K. Cham, "3D model-based pose invariant face
recognition from multiple views," Computer Vision, IET, vol.1, i.1, pp.
25 - 34, March 2007.
[5] S. Feng, H. Krim, I. Gu, M. Viberg, "3D Face Recognition Using Affine
Integral Invariants," Proc. IEEE Int-l Conf. Acoustics, Speech and Signal
Processing (ICASSP) 2006, vol.2, pp. 14-19.
[6] A.F. Abate, M. Nappi, D. Riccio, G. Sabatino, "3D Face Recognition
using Normal Sphere and General Fourier Descriptor," Proc. 18th Int-l
Conf. Pattern Recognition (ICPR), 2006, vol.3, pp. 1183-1186.
[7] L. Wang, L. Ding, X. Ding, and C. Fang, "Improved 3D assisted poseinvariant
face recognition, Proc. ICASSP 2009, pp. 889 - 892.
[8] X. Liu; T. Chen, "Pose-robust face recognition using geometry assisted
probabilistic modeling," Proc. IEEE Conf. Computer Vision and Pattern Recognition, (CVPR) 2005, vol.1, pp. 502 - 509.
[9] T. Vetter and T. Poggio, "Linear object classes and image synthesis from
a single example image," PAMI, vol.19, no.7, pp.733-742, 1997.
[10] T. Vetter, "Synthesis of novel views from a single face image," Int-l J.
Computer Vision (IJCV), vol.28, no.2, 1998, pp.103-116.
[11] X. Chai, S. Shan, X. Chen and W. Gao, "Locally Linear Regression for
Pose-Invariant Face Recognition," IEEE Trans. Image Processing, vol.
16, Iss.7, pp. 1716-1725, 2007.
[12] A.V. Nefian and M.H. Hayes III, "Hidden Markov Models for Face
Recognition," Proc. ICASSP, 1998, pp. 2721-2724.
[13] A.V. Nefian and M.H. Hayes III, "An Embedded HMM Based Approach
for Face Detection and Recognition," Proc. ICASSP, vol.6, 1999, pp.
3553-3556.
[14] S. Eickeler, S. M├╝ller, and G. Rigoll, "Improved Face Recognition using
Pseudo 2-D Hidden Markov Models," in Workshop on Advances in Facial
Image Analysis and Recognition Technology (AFIART), Freiburg,
Germany, 1998.
[15] F. Samaria and S. Young, "HMM-based architecture for face identification,"
Image and Vision Computing, 12(8), pp. 537-543, 1994.
[16] P.J. Phillips, "Support Vector Machines Applied to Face Recognition," in
Advances in Neural Information Processing Systems, vol.11, M.J.
Kearns et. al., eds., MIT Press, 1999.
[17] K. Jonsson, J. Matas, J. Kittler, and Y.P. Li, "Learning Support Vectors
for Face Verification and Recognition," Proc. IEEE Int-l Conf. on Automatic
Face and Gesture Recognition (FG), 2000, pp. 208-213.
[18] B. Heisele, P. Ho, and T. Poggio, "Face Recognition with Support Vector
Machines: Global versus Component-Based Approach," Computer Vision
and Image Understanding (CVIU), vol.91, no. 1/2, pp. 6-21, 2003.
[19] B. Heisele, T. Serre and T. Poggio, "A Component-based Framework for
Face Detection and Identification," IJCV, vol.74, no.2, pp. 167-181,
2007.
[20] P.H. Lee, Y.W. Wang, J. Hsu and Y.P. Hung, "Facial Features Extracted
by 2-D HMM for Face Recognition with Pose Variations," Proc. of IAPR
Conference on Machine Vision Applications (MVA), pp. 392~395, 2007.
[21] L. R. Rabiner, "A Tutorial on Hidden Markov Models and Se-lected
Applications in Speech Recognition," Proc. of IEEE, vol. 77, no. 2, pp.
257-286, 1989.
[22] C.J.C. Burges, "Simplified Support Vector Decision Rules," Proc. 13th
Int Conf. on Machine Learning, pp. 71-78, 1996.
[23] T. Sim, S. Baker, and M. Bsat, "CMU pose illumination and expression(
PIE) database," PAMI, IEEE Trans, vol.25, NO.12, Dec 2003. pp.
1613 - 1618, 2003.
[24] Frgc P.J. Phillips, P.J. Flynn, T. Scruggs, K.W. Bowyer, J. Chang, K.
Hoffman, J. Marques, M. Jaesik, W. Worek, "Overview of the Face Recognition
Grand Challenge," CVPR 2005, vol.1, 20-25, pp.947-954.
[25] P.J. Phillips, P.J. Flynn, T. Scruggs, K.W. Bowyer, W. Worek, " Preliminary
Face Recognition Grand Challenge Results," Proc. 7th Int-l Conf Automatic
Face and Gesture Recognition, pp. 15-24, 2006.
[26] A.M. Martinez and R. Benavente, "The AR Face Database," CVC Technical
Report #24 , June 1998.
[27] K. Messar, J. Matas, and J. Kittler, "XM2VTSDB: The Extended M2VTS
Database, " Proc. 2nd Int-l Conf. Audio and Video-based Biometric Person Authentication
(AVBPA-99-), pp. 2 - 14, 1999.