Among neural models the Support Vector Machine
(SVM) solutions are attracting increasing attention, mostly because
they eliminate certain crucial questions involved by neural network
construction. The main drawback of standard SVM is its high
computational complexity, therefore recently a new technique, the
Least Squares SVM (LS–SVM) has been introduced. In this paper we
present an extended view of the Least Squares Support Vector
Regression (LS–SVR), which enables us to develop new
formulations and algorithms to this regression technique. Based on
manipulating the linear equation set -which embodies all information
about the regression in the learning process- some new methods are
introduced to simplify the formulations, speed up the calculations
and/or provide better results.
[1] V. Vapnik, "The Nature of Statistical Learning Theory", New-York:
Springer-Verlag., 1995
[2] J. A. K. Suykens, V. T. Gestel, J. De Brabanter, B. De Moor, J.
Vandewalle, "Least Squares Support Vector Machines", World
Scientific, 2002
[3] J. A. K. Suykens, L. Lukas, and J. Vandewalle, "Sparse approximation
using least squares support vector machines", IEEE International
Symposium on Circuits and Systems ISCAS'2000, 2000
[4] J. A. K. Suykens, L. Lukas, and J. Vandewalle, "Sparse least squares
support vector machine classifiers", ESANN'2000 European Symposium
on Artificial Neural Networks, 2000, pp. 37-42.
[5] J.A.K. Suykens, J. De Brabanter, L. Lukas, and J. Vandewalle,
"Weighted least squares support vector machines: robustness and sparse
approximation", Neurocomputing, 2002. pp. 85-105
[6] B. Schölkopf and A. Smola: Learning with Kernels. Support Vector
Machines, Regularization, Optimization, and Beyond. The MIT Press,
Cambridge, MA, 2002.
[7] J. Valyon and G. Horv├íth, ÔÇ×A generalized LS-SVM", SYSID'2003
Rotterdam, 2003, pp. 827-832.
[8] J. Valyon and G. Horv├íth, ÔÇ×A Sparse Least Squares Support Vector
Machine Classifier", Proceedings of the International Joint Conference
on Neural Networks IJCNN 2004, 2004, pp. 543-548.
[9] P. W. Holland, and R. E. Welsch, "Robust Regression Using Iteratively
Reweighted Least-Squares," Communications in Statistics: Theory and
Methods, A6, 1977, pp. 813-827.
[10] P. J. Huber, Robust Statistics, Wiley, 1981.
[11] B. Schölkopf, S. Mika, C.J.C. Burges, P. Knirsch, K.-R. M├╝ller, G.
R├ñtsch, and A. Smola, ÔÇ×Input space vs. feature space in kernel-based
methods". IEEE Transactions on Neural Networks, 1999, 10(5), pp.
1000-1017.
[12] G. Baudat and F. Anouar, "Kernel-based methods and function
approximation". In International Joint Conference on Neural Networks,
pages 1244-1249, Washington DC, 2001. July 15-19.
[13] Yuh - Jye Lee and O. L. Mangasarian, "RSVM: Reduced Support
Vector Machines", Proceedings of the First SIAM International
Conference on Data Mining, Chicago, 2001. April 5-7.
[14] W. H. Press, S. A. Teukolsky, W. T. Wetterling and B. P. Flannery ,
"Numerical Recipes in C", Cambridge University Press, Books On-Line,
Available: www.nr.com, 2002
[15] H. Golub and Charles F. Van Loan, Matrix Computations", Gene Johns
Hopkins University Press, 1989
[1] V. Vapnik, "The Nature of Statistical Learning Theory", New-York:
Springer-Verlag., 1995
[2] J. A. K. Suykens, V. T. Gestel, J. De Brabanter, B. De Moor, J.
Vandewalle, "Least Squares Support Vector Machines", World
Scientific, 2002
[3] J. A. K. Suykens, L. Lukas, and J. Vandewalle, "Sparse approximation
using least squares support vector machines", IEEE International
Symposium on Circuits and Systems ISCAS'2000, 2000
[4] J. A. K. Suykens, L. Lukas, and J. Vandewalle, "Sparse least squares
support vector machine classifiers", ESANN'2000 European Symposium
on Artificial Neural Networks, 2000, pp. 37-42.
[5] J.A.K. Suykens, J. De Brabanter, L. Lukas, and J. Vandewalle,
"Weighted least squares support vector machines: robustness and sparse
approximation", Neurocomputing, 2002. pp. 85-105
[6] B. Schölkopf and A. Smola: Learning with Kernels. Support Vector
Machines, Regularization, Optimization, and Beyond. The MIT Press,
Cambridge, MA, 2002.
[7] J. Valyon and G. Horv├íth, ÔÇ×A generalized LS-SVM", SYSID'2003
Rotterdam, 2003, pp. 827-832.
[8] J. Valyon and G. Horv├íth, ÔÇ×A Sparse Least Squares Support Vector
Machine Classifier", Proceedings of the International Joint Conference
on Neural Networks IJCNN 2004, 2004, pp. 543-548.
[9] P. W. Holland, and R. E. Welsch, "Robust Regression Using Iteratively
Reweighted Least-Squares," Communications in Statistics: Theory and
Methods, A6, 1977, pp. 813-827.
[10] P. J. Huber, Robust Statistics, Wiley, 1981.
[11] B. Schölkopf, S. Mika, C.J.C. Burges, P. Knirsch, K.-R. M├╝ller, G.
R├ñtsch, and A. Smola, ÔÇ×Input space vs. feature space in kernel-based
methods". IEEE Transactions on Neural Networks, 1999, 10(5), pp.
1000-1017.
[12] G. Baudat and F. Anouar, "Kernel-based methods and function
approximation". In International Joint Conference on Neural Networks,
pages 1244-1249, Washington DC, 2001. July 15-19.
[13] Yuh - Jye Lee and O. L. Mangasarian, "RSVM: Reduced Support
Vector Machines", Proceedings of the First SIAM International
Conference on Data Mining, Chicago, 2001. April 5-7.
[14] W. H. Press, S. A. Teukolsky, W. T. Wetterling and B. P. Flannery ,
"Numerical Recipes in C", Cambridge University Press, Books On-Line,
Available: www.nr.com, 2002
[15] H. Golub and Charles F. Van Loan, Matrix Computations", Gene Johns
Hopkins University Press, 1989
@article{"International Journal of Information, Control and Computer Sciences:57942", author = "József Valyon and Gábor Horváth", title = "Extended Least Squares LS–SVM", abstract = "Among neural models the Support Vector Machine
(SVM) solutions are attracting increasing attention, mostly because
they eliminate certain crucial questions involved by neural network
construction. The main drawback of standard SVM is its high
computational complexity, therefore recently a new technique, the
Least Squares SVM (LS–SVM) has been introduced. In this paper we
present an extended view of the Least Squares Support Vector
Regression (LS–SVR), which enables us to develop new
formulations and algorithms to this regression technique. Based on
manipulating the linear equation set -which embodies all information
about the regression in the learning process- some new methods are
introduced to simplify the formulations, speed up the calculations
and/or provide better results.", keywords = "Function estimation, Least–Squares Support VectorMachines, Regression, System Modeling", volume = "1", number = "12", pages = "3920-9", }