Self Organizing Mixture Network in Mixture Discriminant Analysis: An Experimental Study

In the recent works related with mixture discriminant analysis (MDA), expectation and maximization (EM) algorithm is used to estimate parameters of Gaussian mixtures. But, initial values of EM algorithm affect the final parameters- estimates. Also, when EM algorithm is applied two times, for the same data set, it can be give different results for the estimate of parameters and this affect the classification accuracy of MDA. Forthcoming this problem, we use Self Organizing Mixture Network (SOMN) algorithm to estimate parameters of Gaussians mixtures in MDA that SOMN is more robust when random the initial values of the parameters are used [5]. We show effectiveness of this method on popular simulated waveform datasets and real glass data set.

Authors:

• Nazif Çalış
• Murat Erişoğlu
• Hamza Erol
• Tayfun Servi

Keywords:

• Self Organizing Mixture Network
• MixtureDiscriminant Analysis
• Waveform Datasets
• Glass Identification
• Mixture of Multivariate Normal Distributions

References:

[1] Hastie T. and Tibshirani R. (1996), Discriminant Analysis by Gaussian
Mixtures, Journal of the Royal Statistical Society. Series B
(Methodological), Vol. 58, No. 1, pp. 155-176
[2] Zohar Halbe, Mayer Aladjem(2005),Model-based mixture discriminant
analysisÔÇöan experimentalstudy, Pattern Recognition, 38, 437-440
[3] Bashir S. and Carter E. M. (2005),Robust Reduced Rank Mixture
Discriminant Analysis, Communications in Statistics Theory and
Methods, 34, 135-145
[4] Bashir S. and Carter E.M. (2005),High breakdown mixture discriminant
analysis, Journal of Multivariate Analysis, 93, 102-11
[5] Yin H. and Allinson N. M. (2001),Self-Organizing Mixture Networks
for Probability Density Estimation, IEEE Transactions on Neural
Networks, Vol. 12, No. 2, pp. 405-411
[6] P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from
incomplete date via the EM algorithm," J. Roy. Statist. Soc. B, vol.39,
pp. 1-38, 1977.
[7] S. Kullback and R. A. Leibler, "On information and sufficiency," Ann.
Math. Statist., vol. 22, pp. 79-86, 1951.
[8] H. Robbins and S. Monro, "A stochastic approximation method," Ann.
Math. Statist., vol. 22, pp. 400-407, 1951.
[9] Murphy, P. M., Aha, D. W. (1995). UCI Repository of Machine
Learning Databases. Irvine, CA: University of California, Dept. of
Information and Computer Science.
[10] Breiman, L., Friedman, J., Olshen, R. and Stone, C. (1984),
Classification and Regression Trees Belmont; Wadsworth
[11] Xu, L. and Jordan, M. I. (1993). Unsupervised learning by EM
algorithm based on finite mixture of Gaussians. In Proc. World
Congress Neural Networks, vol. 2, pp. 431-434, Portland, OR, USA.
[12] McLachlan, G.J., Peel, D., Basford, K.E., and Adams, P. (1999). The
EMMIX software for the fitting of mixtures of normal and tcomponents.
Journal of Statistical Software 4, No. 2.
[13] Yin, H. and Allinson, N. M. (1997). Comparison of a Bayesian SOM
with the EM algorithm for Gaussian mixtures. Proc. Workshop on Self-
Organising Maps (WSOM'97), pp. 118-123.