Increasing The Speed of Convergence of an Artificial Neural Network based ARMA Coefficients Determination Technique

In this paper, novel techniques in increasing the accuracy and speed of convergence of a Feed forward Back propagation Artificial Neural Network (FFBPNN) with polynomial activation function reported in literature is presented. These technique was subsequently used to determine the coefficients of Autoregressive Moving Average (ARMA) and Autoregressive (AR) system. The results obtained by introducing sequential and batch method of weight initialization, batch method of weight and coefficient update, adaptive momentum and learning rate technique gives more accurate result and significant reduction in convergence time when compared t the traditional method of back propagation algorithm, thereby making FFBPNN an appropriate technique for online ARMA coefficient determination.

Authors:

• Abiodun M. Aibinu
• Momoh J. E. Salami
• Amir A. Shafie
• Athaur Rahman Najeeb

Keywords:

• Adaptive Learning rate
• Adaptive momentum
• Autoregressive
• Modeling
• Neural Network.

References:

[1] Z. P. Liang, P. C. Lauterbur, "Principles of Magnetic Resonance Imaging,
A signal processing perspective", IEEE Press, New York, 2000.
[2] A. M. Aibinu, M. J.E. Salami, A. A. Shafie and A. R. Najeeb, "
Model Order Determination for MRI Signal", accepted for publication
, International Conference on Medical system Engineering (ICMSE),
2008, Singapore, August - September 2008.
[3] D. G. Nishimura, "Principles of Magnetic Resonance Imaging", April
1996.
[4] M. R. Smith, S. T. Nichols, R. M. Henkelman and M. L. Wood,
"Application of Autoregressive Moving Average Parametric Modeling
in Magnetic Resonance Image Reconstruction", IEEE Transactions on
Medical Imaging, Vol. M1-5:3, pp 257 - 261, 1986.
[5] M. R. Smith, S. T. Nichols, R. Constable and R. Henkelman, "A
quantitative comparison of the TERA modeling and DFT magnetic
resonance image reconstruction techniques", Magn. Reson. Med., Vol.
19 pp. 1-19, 1991.
[6] K. H. Chon, R. J. Cohen, "Linear and Non-Linear ARMA Model Parameter
Estimation Using Artificial Neural Network", IEEE Transactions on
BioMedical Engineering, Vol. 44, No 3, pp 168 - 174, 1997.
[7] K. H. Chon, D. Hoyer, A. A Armoundas, N-H Holstein-Rathlou and D.
J Marsh, " Robust Nonlinear Autoregressive Moving Average Model Parameter
Estimation Using Recurrent Artificial Neural Network", Annals
of BioMedical Engineering, Vol. 27, pp 538-547, 1999.
[8] A. M. Aibinu, M. J. E. Salami, A. A. Shafie and A. R. Najeeb
"Performance Evaluation of Autoregressive Moving Average (ARMA)
coefficients determination methods", accepted for publication , International
Conference on Computer System (ICCS) , 2008, Singapore,
August- September 2008.
[9] M. H. Hayes, "Staitical Digital Signal processing and Modelling", John
Wiley & Sons, Canada, 1996.
[10] E. C. Whitman, "The spectral analysis of discrete time series in terms
of linear regressive models", Naval Ordinance Labs Rep., NOLTR-070-
109, White Oak, MD, June 23, 1974.
[11] Z. P. Liang, F. E. Boada, R. T. Constable, E. M. Haacke, P. C.
Lauterbur, and M. R. Smith, "Constrained Reconstruction Methods in
MR Imaging", Reviews of MRM, vol. 4, pp.67 - 185, 1992.
[12] E. Hackle and Z. Liang, "Superresolution Reconstruction Through
Object Modeling and Estimation", IEEE transactions in A.S.S.P, 37:
592 - 595, 1989.
[13] R. Palaniappan, "Towards Optimal Model Oreder Selection for Autoregressive
Spectral Analysis of Mental Tasks Using Genetic Algorithm",
IJCSNS International Journal of Computer Science and Network Security,
Vol. 6 No. 1A, January 2006.
[14] C. C. Yu, Bin-Da Liu "A Simple Procedure in Back Propagation
Trainning", IEEE Trans. Autom. Control, vol. AC-19, pp. 529-535,
2001.
[15] C. C. Yu, Bin-Da Liu "A Backpropapgation Algorithm With Adaptive
Learning Rate and Momentum coefficients", IEEE Trans. Autom.
Control, pp. 1218-1223, 2002.
[16] S. Haykin, " Neural Networks: A comprehensive foundation, 2nd ed.",
Eaglewood, Cliffs, NJ: Prentice Hall.
[17] D. Nguyen, B. Widrow, " Improving the learning speed of 2- Layer
Neural Network by Choosing initial values of the adaptive weights "
Proc. Int. Joint Conference on Neural Networks, Vol. 3, pp.21-26, July,
1990.
[18] J. Rissanen, "Modelling by shortest data description", Automatica,
vol.14, pp.465-471, 1978.
[19] M.J. Salami, A. R. Najeeb, O. Khalifa, K. Arrifin, "MR Reconsturction
with Autoregressive Moving Average", International Conference on
Biotechnology Engineering, Kuala Lumpur, pp 676 - 704, May, 2007.