An Exact Solution to Support Vector Mixture

This paper presents a new version of the SVM mixture algorithm initially proposed by Kwok for classification and regression problems. For both cases, a slight modification of the mixture model leads to a standard SVM training problem, to the existence of an exact solution and allows the direct use of well known decomposition and working set selection algorithms. Only the regression case is considered in this paper but classification has been addressed in a very similar way. This method has been successfully applied to engine pollutants emission modeling.

Authors:

• Monjed Ezzeddinne
• Nicolas Lefebvre
• Régis Lengellé

Keywords:

• Identification
• Learning systems
• Mixture ofExperts
• Support Vector Machines.

References:

[1] Collobert R. and Bengio S., SVMTorch: "Support Vector Machine for
Large-Scale Regression and Classification Problems", Journal of
Machine Learning Research, 1: pp. 143-160, 2001.
[2] Genton M., "Classes of Kernels for Machine Learning: A statistics
Perspective", Journal of Machine Learning Research 2: pp. 299-312, 2001.
[3] Jacobs R., Jordan M., Nowlan S., and Hinton G., "Adaptative Mixtures
of Local Experts", Neural Computation, 3(1): pp. 79 -87, 1991.
[4] Joachims T., Making "Large-Scale SVM Learning Practical", Advances
in Kernel Methods - Support Vector Learning, ch. 11, pp. 169-184 ,
MIT Press, 1999.
[5] Jordan M., Jacobs R., "Hierarchical Mixtures of Experts and the EM
Algorithm", Neural Computation, 6(2): pp. 181-214, 1994.
[6] Kwok J., "Support Vector Mixture for Classification and Regression
Problems", Proceedings of the International Conference on Pattern
Recognition, pp. 255-258, 1998.
[7] Lin C. J., "On the Convergence of the Decomposition Method for
Support Vector Machines", IEEE Transactions on Neural Networks, 12 (2001), pp. 1288-1298.
[8] Osuna E., Freund R. and Girosi F., "An Improved Training Algorithm
for Support Vector Machines", Proceedings of IEEE Workshop on
Neural Networks for Signal Processing pp. 276-285, 1997.
[9] Vapnik V., The Nature of Statistical Learning Theory, Springer, New
York, 1995.
[10] Vapnik V., Statistical Learning Theory, A Wiley-Interscience
Publication, New York, 1998.