An Iterative Algorithm for KLDA Classifier

The Linear discriminant analysis (LDA) can be generalized into a nonlinear form - kernel LDA (KLDA) expediently by using the kernel functions. But KLDA is often referred to a general eigenvalue problem in singular case. To avoid this complication, this paper proposes an iterative algorithm for the two-class KLDA. The proposed KLDA is used as a nonlinear discriminant classifier, and the experiments show that it has a comparable performance with SVM.

Authors:

• D.N. Zheng
• J.X. Wang
• Y.N. Zhao
• Z.H. Yang

Keywords:

• Linear discriminant analysis (LDA)
• kernel LDA (KLDA)
• conjugate gradient algorithm
• nonlinear discriminant classifier.

References:

[1] B. Scholköpf, A. Smola, and K.R. M├╝ller, "Nonlinear Component Analysis
as a Kernel Eigenvalue Problem," Neural Computation, vol.10, pp.
1299-1319, 1998.
[2] G. Baudat and F. Anouar, "Generalized Discriminant Analysis Using a
Kernel Approach," Neural Computation, vol.12, pp. 2385-2404, 2000.
[3] C. Park and H. Park, "Fingerprint Classification Using Nonlinear Discriminant
Analysis," Technical Report, TR 03-034, University of Minnesota,
USA, Sep. 2003.
[4] S. Mika, G. R├ñtsch, J. Weston, B. Schölkopf, and K.R. M├╝ller, "Fisher
discriminant analysis with kernels," Neural Networks for Signal Processing
IX, pp. 41-48, 1999.
[5] P.N. Belhumeour, J.P. Hespanha, and D.J. Kriegman, "Eigenfaces vs.
Fisherfaces: Recognition Using Class Specific Linear Projection," IEEE
Trans. Pattern Analysis and Machine Intell., vol.19, pp. 711-720, 1997.
[6] H.C. Kim, D. Kim, and S.Y. Bang, "Face recognition using LDA mixture
model," Pattern Recognition Letters, vol.24, pp. 2815-2821, 2003.
[7] G. Rätsch, T. Onoda, and K.R. Müller, "Soft Margins for AdaBoost,"
Machine Learning, pp. 1-35, 2000.
[8] C. Burges, "Simplified Support Vector Decision Rules," in Proceedings
of the 13th International Conference on Machine Learning, pp. 71-77,
1996.
[9] B. Schölkopf, P. Knirsch, A. Smola, and C. Burges, "Fast Approximation
of Support Vector Kernel Expansions, and an Interpretation of
Clustering as Approximation in Feature Spaces," Proceedings of the
DAGM Symposium Mustererkennung, pp. 124-132, 1998.