Investigation of Improved Chaotic Signal Tracking by Echo State Neural Networks and Multilayer Perceptron via Training of Extended Kalman Filter Approach

This paper presents a prediction performance of
feedforward Multilayer Perceptron (MLP) and Echo State Networks
(ESN) trained with extended Kalman filter. Feedforward neural
networks and ESN are powerful neural networks which can track and
predict nonlinear signals. However, their tracking performance
depends on the specific signals or data sets, having the risk of
instability accompanied by large error. In this study we explore this
process by applying different network size and leaking rate for
prediction of nonlinear or chaotic signals in MLP neural networks.
Major problems of ESN training such as the problem of initialization
of the network and improvement in the prediction performance are
tackled. The influence of coefficient of activation function in the
hidden layer and other key parameters are investigated by simulation
results. Extended Kalman filter is employed in order to improve the
sequential and regulation learning rate of the feedforward neural
networks. This training approach has vital features in the training of
the network when signals have chaotic or non-stationary sequential
pattern. Minimization of the variance in each step of the computation
and hence smoothing of tracking were obtained by examining the
results, indicating satisfactory tracking characteristics for certain
conditions. In addition, simulation results confirmed satisfactory
performance of both of the two neural networks with modified
parameterization in tracking of the nonlinear signals.

Authors:

• Farhad Asadi
• S. Hossein Sadati

Keywords:

• Feedforward neural networks
• nonlinear signal prediction
• echo state neural networks approach
• leaking rates
• capacity of neural networks.

References:

[1] R. Laje, D. V. Buonomano, "Robust timing and motor patterns by
taming chaos in recurrent neural networks," Nature neuroscience, 2013,
16(7), pp. 925.
[2] M. Lukoševičius, "A practical guide to applying echo state networks,"
Neural networks: Tricks of the trade: Springer, 2012. pp. 659-86.
[3] F. Asadi, M. J. Mollakazemi, S. A. Atyabi, I. Uzelac, A. Ghaffari,
"Cardiac arrhythmia recognition with robust discrete wavelet-based and
geometrical feature extraction via classifiers of SVM and MLP-BP and
PNN neural networks," in Computing in Cardiology Conference (CinC),
2015 2015, IEEE.
[4] F. Asadi, M. J. Mollakazemi, S. Ghiasi, S. H. Sadati, "Enhancement of
life-threatening arrhythmia discrimination in the intensive care unit with
morphological features and interval feature extraction via random forest
classifier," in Computing in Cardiology Conference (CinC), 2016 2016,
IEEE.
[5] S. A. S. Mousavi, X. Zhang, T. Seigler, J. B. Hoagg, "Characteristics
that make dynamic systems difficult for a human to control," in
American Control Conference (ACC), 2016 2016, IEEE.
[6] F. Matveeva, S. A. S. Mousavi, X. Zhang, T. Seigler, J. B. Hoagg, "On
the effects of changing reference command as humans learn to control
dynamic systems," in Decision and Control (CDC), 2016 IEEE 55th
Conference on 2016, IEEE.
[7] A. Hamidisepehr, M. P. Sama, "A low-cost method for collecting
hyperspectral measurements from a small unmanned aircraft system," in
Autonomous Air and Ground Sensing Systems for Agricultural
Optimization and Phenotyping III 2018, International Society for Optics
and Photonics.
[8] A. Hamidisepehr, M. P. Sama, A. P. Turner, O. O. Wendroth, "A
Method for Reflectance Index Wavelength Selection from Moisture-
Controlled Soil and Crop Residue Samples," Transactions of the
ASABE, 2017, 60(5), pp. 1479-87.
[9] A. Ghaffari, M. J. Mollakazemi, S. A. Atyabi, M. Niknazar, "Robust
fetal QRS detection from noninvasive abdominal electrocardiogram
based on channel selection and simultaneous multichannel processing,"
Australasian physical & engineering sciences in medicine, 2015, 38(4),
pp. 581-92.
[10] M. Mollakazemi, F. Asadi, M. Tajnesaei, A. Ghaffari, "Fetal QRS
Detection in Noninvasive Abdominal Electrocardiograms Using
Principal Component Analysis and Discrete Wavelet Transforms with
Signal Quality Estimation," Journal of Biomedical Physics and
Engineering, 2016, pp.
[11] M. J. Mollakazemi, S. A. Atyabi, A. Ghaffari, "Heart beat detection
using a multimodal data coupling method," Physiological measurement,
2015, 36(8), pp. 1729.
[12] D. Brezak, T. Bacek, D. Majetic, J. Kasac, B. Novakovic, "A
comparison of feed-forward and recurrent neural networks in time series
forecasting," in Computational Intelligence for Financial Engineering &
Economics (CIFEr), 2012 IEEE Conference on 2012, IEEE.
[13] M. Khashei, M. Bijari, "A novel hybridization of artificial neural
networks and ARIMA models for time series forecasting," Applied Soft
Computing, 2011, 11(2), pp. 2664-75.
[14] A. Petrenas, V. Marozas, L. Sörnmo, A. Lukosevicius, "An echo state
neural network for QRST cancellation during atrial fibrillation," IEEE
Transactions on Biomedical Engineering, 2012, 59(10), pp. 2950.
[15] F. Asadi, M. Khorram, S. A. A. Moosavian, "CPG-based gait planning
of a quadruped robot for crossing obstacles," in Robotics and
Mechatronics (ICROM), 2015 3rd RSI International Conference on
2015, IEEE.
[16] F. Asadi, M. Khorram, S. A. A. Moosavian, "CPG-based gait transition
of a quadruped robot," in Robotics and Mechatronics (ICROM), 2015
3rd RSI International Conference on 2015, IEEE.
[17] B. Choubin, S. Khalighi-Sigaroodi, A. Malekian, Ö. Kişi, "Multiple
linear regression, multi-layer perceptron network and adaptive neurofuzzy
inference system for forecasting precipitation based on large-scale
climate signals," Hydrological Sciences Journal, 2016, 61(6), pp. 1001-
9.
[18] D. Sussillo, "Neural circuits as computational dynamical systems,"
Current opinion in neurobiology, 2014, 25pp. 156-63.
[19] D. Sussillo, M. M. Churchland, M. T. Kaufman, K. V. Shenoy, "A
neural network that finds a naturalistic solution for the production of
muscle activity," Nature neuroscience, 2015, 18(7), pp. 1025.
[20] M. Lukoševičius, H. Jaeger, "Reservoir computing approaches to
recurrent neural network training," Computer Science Review, 2009,
3(3), pp. 127-49.
[21] Z. Shi, M. Han, "Support vector echo-state machine for chaotic timeseries
prediction," IEEE Trans Neural Networks, 2007, 18(2), pp. 359-
72.
[22] M. Čerňanský, P. Tiňo, "Predictive modeling with echo state networks,"
in International Conference on Artificial Neural Networks 2008,
Springer.
[23] Y. Xia, B. Jelfs, M. M. Van Hulle, J. C. Príncipe, D. P. Mandic, "An
augmented echo state network for nonlinear adaptive filtering of
complex noncircular signals," IEEE Transactions on Neural Networks,
2011, 22(1), pp. 74-83.
[24] M. J. Mollakazemi, F. Asadi, A. Ghafouri, "The evaluation of the
performance of different filtering approaches in tracking problem and
the effect of noise variance," simulation, 2015, 10pp. 12.
[25] F. Asadi, M. J. Mollakazemi, "Investigation on Performance of Change
Point Algorithm in Time Series Dynamical Regimes and Effect of Data
Characteristics," World Academy of Science, Engineering and
Technology, International Journal of Mechanical, Aerospace, Industrial,
Mechatronic and Manufacturing Engineering, 2015, 8(10), pp. 1787-93.
[26] F. Asadi, M. Mollakazemi, A. Ghafouri, "The influence of parameters of
modeling and data distribution for optimal condition on locally weighted
projection regression method," Accepted and oral presentation in
ICMSE, 2014, pp. 27-8.
[27] M. J. Mollakazemi, F. Asadi, "Real Time Adaptive Obstacle Avoidance
in Dynamic Environments with Different DS," Accepted and oral
presentation in ICARM, 2014, pp. 27-8.