A New Weighted LDA Method in Comparison to Some Versions of LDA

Linear Discrimination Analysis (LDA) is a linear solution for classification of two classes. In this paper, we propose a variant LDA method for multi-class problem which redefines the between class and within class scatter matrices by incorporating a weight function into each of them. The aim is to separate classes as much as possible in a situation that one class is well separated from other classes, incidentally, that class must have a little influence on classification. It has been suggested to alleviate influence of classes that are well separated by adding a weight into between class scatter matrix and within class scatter matrix. To obtain a simple and effective weight function, ordinary LDA between every two classes has been used in order to find Fisher discrimination value and passed it as an input into two weight functions and redefined between class and within class scatter matrices. Experimental results showed that our new LDA method improved classification rate, on glass, iris and wine datasets, in comparison to different versions of LDA.

Authors:

• Delaram Jarchi
• Reza Boostani

Keywords:

• Discriminant vectors
• weighted LDA
• uncorrelation
• principle components
• Fisher-face method
• Bootstarp method.

References:

[1] Xiao-Yuan Jing, David Zhang, and Yuan-Yan Tang ," An improved
LDA Approach", IEEE Transaction on Syatems, Man, And
CyberneticsÔÇöPart B: Cybernetics, VOL. 34, NO. 5, October 2004.
[2] Yu Bing. Jin Lianfu. Chen Ping,"A new LDA-based method for face
recognition", Proceedings of 16th International Conference on Pattern
Recognition, Volume 1, 11-15 Aug. 2002 Page(s):168 - 171 vol.. 1.
[3] Tang, E.K. Suganthan, P.N. Yao, X, "Generalized LDA Using
Relevance Weighting and evolution strategy", Congress on Evolutionary
Computation, 2004. CEC2004. Volume 2, 19-23 June 2004 Page(s):
2230 - 2234 Vol. 2.
[4] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, "Eigenfaces vs.
fisherface: Recognition using class specific linear projection," IEEE
Trans. Pattern Anal. Machine Intell., vol. 19, pp. 711-720, July 1997.
[5] M. Loog, R.P.W. Duin and R. Haeb-Umbach, ''Multiclass linear
dimension reduction by weighted pairwise fisher criteria", IEEE
Transaction on Pattern Analysis and Machine
Intelligence,23(7),2001,00.762-766.
[6] A. M. Martinez and A. C. Kak, "PCA versus LDA," IEEE Trans. Pattern
Anal. Machine Intell., vol. 23, pp. 228-233, Feb. 2001.
[7] Chernick M.R., Bootstrap Methods: A Practitioner-s Guide, John Wiley
and Sons, New York, 1999.
[8] Goldberg, D.E. "Genetic algorithms as a computational theory of
conceptual design". In Applications of Artificial Intelligence in
Engineenng. Vol. 6, 1991, pp, 3-16.
[9] David E. Goldberg, "Genetic Algorithms in Search, Optimization and
Machine Learning", Addison-Wesley Longman Publishing Co., Inc.,
Boston, MA, 1989.