Application of Extreme Learning Machine Method for Time Series Analysis
In this paper, we study the application of Extreme
Learning Machine (ELM) algorithm for single layered feedforward
neural networks to non-linear chaotic time series problems. In this
algorithm the input weights and the hidden layer bias are randomly
chosen. The ELM formulation leads to solving a system of linear
equations in terms of the unknown weights connecting the hidden
layer to the output layer. The solution of this general system of
linear equations will be obtained using Moore-Penrose generalized
pseudo inverse. For the study of the application of the method we
consider the time series generated by the Mackey Glass delay
differential equation with different time delays, Santa Fe A and
UCR heart beat rate ECG time series. For the choice of sigmoid,
sin and hardlim activation functions the optimal values for the
memory order and the number of hidden neurons which give the
best prediction performance in terms of root mean square error are
determined. It is observed that the results obtained are in close
agreement with the exact solution of the problems considered
which clearly shows that ELM is a very promising alternative
method for time series prediction.
[1] P. J. Brockwell and R. A. Davis, "Introduction to Time Series
Forecasting", 2nd ed., Springer, Berlin, 2002.
[2] M.Casdagli, "Nonlinear Prediction of Chaotic Time Series", Physica D,
35, (1989), pp. 335-356.
[3] K. Y. Chen and C. H. Wang, "A Hybrid SARIMA and Support Vector
Machines for Forecasting the Production Values of the Machinery
Industry in Taiwan", Expert Systems with Applications, (2006).
[4] Y. Chen, B.Yang and J.Dong, "Time Series Prediction using a Local
Linear Wavelet Neural Network", Neurocomputing, 69 (2006), pp.449-
465.
[5] G. B. Huang, Q. Y. Zhu and C. K. Siew, "Extreme Learning Machine:
Theory and Applications", Neurocomputing, 70, (2006), pp.489-501.
[6] R.Malhotra and D.K.Malhotra, "Evaluating Consumer Loans Using
Neural Networks", Omega, 31, (2003), pp.83-96.
[7] N.Mani and P.Voumard, "An Optical Character Recognition Using
Artificial Neural Network", IEEE Int. Conf. on Systems, Man, and
Cybernetics, Vol. 3, (1996), pp.2244-2247.
[8] S.Mukherjee, E.Osuna and F.Girosi, "Nonlinear Prediction of Chaotic
Time Series Using Support Vector Machines", in Neural Networks for
Signal Processing VII, Proceed. of the IEEE Signal Processing Society
Workshop, FL, (1997), pp.511-520.
[9] K.R.Muller, A.J.Smola, G.Ratsch, B.Schlkopf and J.Kohlmorgen,
"Using Support Vector Machines for Time Series Prediction", in
B.Schlkopf, C.J.C. Burges and A.J.Smola (Eds), Advances in Kernel
Methods- Support Vector Learning, MIT Press, Cambridge, MA, (1999),
pp.243-254.
[10] C.R.Rao and S.K.Mitra, Generalized Inverse of Matrices and its
Applications, Wiley, New York, (1971).
[11] H.A.Rowley, S.Baluja and T.Kanade, "Neural Network based Face
Detection", IEEE Trans. on Pattern Analysis and Machine Intelligence,
Vol.20, No. 1, (1998), pp.23-38.
[12] Z.Tang, P.A.Fishwick, "Feedforward Neural Nets as Models for Time
Series Forecasting", ORSA J. Comput. 5(1993), pp.374-385.
[13] F.E.H.Tay and L.Cao, "Application of Support Vector Machines in
Financial Time Series Forecasting", Omega 29 (2001), pp.309-317.
[14] Q.Tong, H.Zheng and X.Wang, "Gene Prediction Algorithm Based on
the Statistical Combination and the Classification in terms of Gene
Characteristics", Int. Conf. on Neural Networks and Brain, Vol.2,
(2005), pp.673 - 677.
[15] T.B.Trafalis, H.Ince, "Support Vector Machine for Regression and
Applications to Financial Forecasting", Proceedings of the IEEE
INNSENNS Int. Joint Conf., Vol.16, IEEE (2000), pp. 348-353.
[16] F.M.Tseng, H.C.Yu and G.H.Tzeng, "Combining Neural Network
Model with Seasonal Time Series ARIMA Model", Technological
Forecasting and Social Change, 69 (2002), pp.71-87.
[17] G.P.Zhang, E.B.Patuwo and M.Y.Hu, "A Simulation Study of Artificial
Neural Networks for Nonlinear Time Series Forecasting",
Comput.Oper.Res. 28,(2001), pp.381-396.
[18] G.P.Zhang, "Time Series Forecasting using a Hybrid ARIMA and
Neural Network Model", Neurocomputing, 50, (2003),pp. 159-175.
[1] P. J. Brockwell and R. A. Davis, "Introduction to Time Series
Forecasting", 2nd ed., Springer, Berlin, 2002.
[2] M.Casdagli, "Nonlinear Prediction of Chaotic Time Series", Physica D,
35, (1989), pp. 335-356.
[3] K. Y. Chen and C. H. Wang, "A Hybrid SARIMA and Support Vector
Machines for Forecasting the Production Values of the Machinery
Industry in Taiwan", Expert Systems with Applications, (2006).
[4] Y. Chen, B.Yang and J.Dong, "Time Series Prediction using a Local
Linear Wavelet Neural Network", Neurocomputing, 69 (2006), pp.449-
465.
[5] G. B. Huang, Q. Y. Zhu and C. K. Siew, "Extreme Learning Machine:
Theory and Applications", Neurocomputing, 70, (2006), pp.489-501.
[6] R.Malhotra and D.K.Malhotra, "Evaluating Consumer Loans Using
Neural Networks", Omega, 31, (2003), pp.83-96.
[7] N.Mani and P.Voumard, "An Optical Character Recognition Using
Artificial Neural Network", IEEE Int. Conf. on Systems, Man, and
Cybernetics, Vol. 3, (1996), pp.2244-2247.
[8] S.Mukherjee, E.Osuna and F.Girosi, "Nonlinear Prediction of Chaotic
Time Series Using Support Vector Machines", in Neural Networks for
Signal Processing VII, Proceed. of the IEEE Signal Processing Society
Workshop, FL, (1997), pp.511-520.
[9] K.R.Muller, A.J.Smola, G.Ratsch, B.Schlkopf and J.Kohlmorgen,
"Using Support Vector Machines for Time Series Prediction", in
B.Schlkopf, C.J.C. Burges and A.J.Smola (Eds), Advances in Kernel
Methods- Support Vector Learning, MIT Press, Cambridge, MA, (1999),
pp.243-254.
[10] C.R.Rao and S.K.Mitra, Generalized Inverse of Matrices and its
Applications, Wiley, New York, (1971).
[11] H.A.Rowley, S.Baluja and T.Kanade, "Neural Network based Face
Detection", IEEE Trans. on Pattern Analysis and Machine Intelligence,
Vol.20, No. 1, (1998), pp.23-38.
[12] Z.Tang, P.A.Fishwick, "Feedforward Neural Nets as Models for Time
Series Forecasting", ORSA J. Comput. 5(1993), pp.374-385.
[13] F.E.H.Tay and L.Cao, "Application of Support Vector Machines in
Financial Time Series Forecasting", Omega 29 (2001), pp.309-317.
[14] Q.Tong, H.Zheng and X.Wang, "Gene Prediction Algorithm Based on
the Statistical Combination and the Classification in terms of Gene
Characteristics", Int. Conf. on Neural Networks and Brain, Vol.2,
(2005), pp.673 - 677.
[15] T.B.Trafalis, H.Ince, "Support Vector Machine for Regression and
Applications to Financial Forecasting", Proceedings of the IEEE
INNSENNS Int. Joint Conf., Vol.16, IEEE (2000), pp. 348-353.
[16] F.M.Tseng, H.C.Yu and G.H.Tzeng, "Combining Neural Network
Model with Seasonal Time Series ARIMA Model", Technological
Forecasting and Social Change, 69 (2002), pp.71-87.
[17] G.P.Zhang, E.B.Patuwo and M.Y.Hu, "A Simulation Study of Artificial
Neural Networks for Nonlinear Time Series Forecasting",
Comput.Oper.Res. 28,(2001), pp.381-396.
[18] G.P.Zhang, "Time Series Forecasting using a Hybrid ARIMA and
Neural Network Model", Neurocomputing, 50, (2003),pp. 159-175.
@article{"International Journal of Information, Control and Computer Sciences:61183", author = "Rampal Singh and S. Balasundaram", title = "Application of Extreme Learning Machine Method for Time Series Analysis", abstract = "In this paper, we study the application of Extreme
Learning Machine (ELM) algorithm for single layered feedforward
neural networks to non-linear chaotic time series problems. In this
algorithm the input weights and the hidden layer bias are randomly
chosen. The ELM formulation leads to solving a system of linear
equations in terms of the unknown weights connecting the hidden
layer to the output layer. The solution of this general system of
linear equations will be obtained using Moore-Penrose generalized
pseudo inverse. For the study of the application of the method we
consider the time series generated by the Mackey Glass delay
differential equation with different time delays, Santa Fe A and
UCR heart beat rate ECG time series. For the choice of sigmoid,
sin and hardlim activation functions the optimal values for the
memory order and the number of hidden neurons which give the
best prediction performance in terms of root mean square error are
determined. It is observed that the results obtained are in close
agreement with the exact solution of the problems considered
which clearly shows that ELM is a very promising alternative
method for time series prediction.", keywords = "Chaotic time series, Extreme learning machine,
Generalization performance.", volume = "1", number = "11", pages = "3630-7", }