Application of Extreme Learning Machine Method for Time Series Analysis

In this paper, we study the application of Extreme Learning Machine (ELM) algorithm for single layered feedforward neural networks to non-linear chaotic time series problems. In this algorithm the input weights and the hidden layer bias are randomly chosen. The ELM formulation leads to solving a system of linear equations in terms of the unknown weights connecting the hidden layer to the output layer. The solution of this general system of linear equations will be obtained using Moore-Penrose generalized pseudo inverse. For the study of the application of the method we consider the time series generated by the Mackey Glass delay differential equation with different time delays, Santa Fe A and UCR heart beat rate ECG time series. For the choice of sigmoid, sin and hardlim activation functions the optimal values for the memory order and the number of hidden neurons which give the best prediction performance in terms of root mean square error are determined. It is observed that the results obtained are in close agreement with the exact solution of the problems considered which clearly shows that ELM is a very promising alternative method for time series prediction.




References:
[1] P. J. Brockwell and R. A. Davis, "Introduction to Time Series
Forecasting", 2nd ed., Springer, Berlin, 2002.
[2] M.Casdagli, "Nonlinear Prediction of Chaotic Time Series", Physica D,
35, (1989), pp. 335-356.
[3] K. Y. Chen and C. H. Wang, "A Hybrid SARIMA and Support Vector
Machines for Forecasting the Production Values of the Machinery
Industry in Taiwan", Expert Systems with Applications, (2006).
[4] Y. Chen, B.Yang and J.Dong, "Time Series Prediction using a Local
Linear Wavelet Neural Network", Neurocomputing, 69 (2006), pp.449-
465.
[5] G. B. Huang, Q. Y. Zhu and C. K. Siew, "Extreme Learning Machine:
Theory and Applications", Neurocomputing, 70, (2006), pp.489-501.
[6] R.Malhotra and D.K.Malhotra, "Evaluating Consumer Loans Using
Neural Networks", Omega, 31, (2003), pp.83-96.
[7] N.Mani and P.Voumard, "An Optical Character Recognition Using
Artificial Neural Network", IEEE Int. Conf. on Systems, Man, and
Cybernetics, Vol. 3, (1996), pp.2244-2247.
[8] S.Mukherjee, E.Osuna and F.Girosi, "Nonlinear Prediction of Chaotic
Time Series Using Support Vector Machines", in Neural Networks for
Signal Processing VII, Proceed. of the IEEE Signal Processing Society
Workshop, FL, (1997), pp.511-520.
[9] K.R.Muller, A.J.Smola, G.Ratsch, B.Schlkopf and J.Kohlmorgen,
"Using Support Vector Machines for Time Series Prediction", in
B.Schlkopf, C.J.C. Burges and A.J.Smola (Eds), Advances in Kernel
Methods- Support Vector Learning, MIT Press, Cambridge, MA, (1999),
pp.243-254.
[10] C.R.Rao and S.K.Mitra, Generalized Inverse of Matrices and its
Applications, Wiley, New York, (1971).
[11] H.A.Rowley, S.Baluja and T.Kanade, "Neural Network based Face
Detection", IEEE Trans. on Pattern Analysis and Machine Intelligence,
Vol.20, No. 1, (1998), pp.23-38.
[12] Z.Tang, P.A.Fishwick, "Feedforward Neural Nets as Models for Time
Series Forecasting", ORSA J. Comput. 5(1993), pp.374-385.
[13] F.E.H.Tay and L.Cao, "Application of Support Vector Machines in
Financial Time Series Forecasting", Omega 29 (2001), pp.309-317.
[14] Q.Tong, H.Zheng and X.Wang, "Gene Prediction Algorithm Based on
the Statistical Combination and the Classification in terms of Gene
Characteristics", Int. Conf. on Neural Networks and Brain, Vol.2,
(2005), pp.673 - 677.
[15] T.B.Trafalis, H.Ince, "Support Vector Machine for Regression and
Applications to Financial Forecasting", Proceedings of the IEEE
INNSENNS Int. Joint Conf., Vol.16, IEEE (2000), pp. 348-353.
[16] F.M.Tseng, H.C.Yu and G.H.Tzeng, "Combining Neural Network
Model with Seasonal Time Series ARIMA Model", Technological
Forecasting and Social Change, 69 (2002), pp.71-87.
[17] G.P.Zhang, E.B.Patuwo and M.Y.Hu, "A Simulation Study of Artificial
Neural Networks for Nonlinear Time Series Forecasting",
Comput.Oper.Res. 28,(2001), pp.381-396.
[18] G.P.Zhang, "Time Series Forecasting using a Hybrid ARIMA and
Neural Network Model", Neurocomputing, 50, (2003),pp. 159-175.