Probability Density Estimation Using Advanced Support Vector Machines and the Expectation Maximization Algorithm

This paper presents a new approach for the prob-ability density function estimation using the Support Vector Ma-chines (SVM) and the Expectation Maximization (EM) algorithms.In the proposed approach, an advanced algorithm for the SVM den-sity estimation which incorporates the Mean Field theory in the learning process is used. Instead of using ad-hoc values for the para-meters of the kernel function which is used by the SVM algorithm,the proposed approach uses the EM algorithm for an automatic optimization of the kernel. Experimental evaluation using simulated data set shows encouraging results.

• Refaat M Mohamed
• Ayman El-Baz
• Aly A. Farag

• Density Estimation
• SVM
• Learning Algorithms
• Parameters Estimation.

<p>[1] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. 2nd ed., Wiley: New York, 2000.
[2] V. Vapnik, The Nature of Statistical Learning Theory. 2nd ed., Springer: New York, 2001.
[3] Refaat M. Mohamed and Aly A. Farag, "Mean Field Theory for Density Estimation Using Support Vector Machines," Seventh International Conference on Information Fusion, Stockholm, July, 2004, pp. 495-501.
[4] V. Vapnik, S. Golowich and A. Smola, "Support Vector Method for Multivariate Density Estimation," Proc. Neural Information Processing Systems 1999 (NIPS 99), Vol. 9, MIT Press:Cambridge, MA, 2000.
[5] B. Scholkopf , C. Burges, and A. Smola, Advances in Kernel Methods:Support Vector Learning. MIT Press:Cambridge, MA, 1999.
[6] Manfred Opper and OleWinther, "Gaussian Processes for Classification: Mean Field Algorithms," Neural Computation., Vol. 12, pp. 2655-2684, 2000.
[7] John W. Lamperti, Probability-A survey of the Mathematical Theory. Wiley Series in probability and Statistics, New York, 1996.</p>