Evaluation of the ANN Based Nonlinear System Models in the MSE and CRLB Senses

The System Identification problem looks for a suitably parameterized model, representing a given process. The parameters of the model are adjusted to optimize a performance function based on error between the given process output and identified process output. The linear system identification field is well established with many classical approaches whereas most of those methods cannot be applied for nonlinear systems. The problem becomes tougher if the system is completely unknown with only the output time series is available. It has been reported that the capability of Artificial Neural Network to approximate all linear and nonlinear input-output maps makes it predominantly suitable for the identification of nonlinear systems, where only the output time series is available. [1][2][4][5]. The work reported here is an attempt to implement few of the well known algorithms in the context of modeling of nonlinear systems, and to make a performance comparison to establish the relative merits and demerits.

Authors:

• M.V Rajesh
• Archana R
• A Unnikrishnan
• R Gopikakumari
• Jeevamma Jacob

Keywords:

• Multilayer neural networks
• Radial Basis Functions
• Clustering algorithm
• Back Propagation training
• Extended Kalmanfiltering
• Mean Square Error
• Nonlinear Modeling
• Cramer RaoLower Bound.

References:

[1] N.K Sinha and B Kuszta, Modeling and Identification of Dynamic
systems,Van Nostrand Reinhold Company, New York,1983.
[2] Simon Haykin, Neural Networks a comprehensive Foundation,
Prentice Hall International Editions, 1999.
[3] A.V. Balakrishnan, Kalman Filtering Theory, Optimization
Software Inc. Publications Division, Newyork,1992
[4] Fa-Long Luo and Rolf Unbehauen, Applied Neural Networks for
Signal Processing, Cambridge University Press, 1997.
[5] Yaakov Bar-Shalaom and Xiao-Rong Li, Estimation and Tracking:
Principles, Techniques and Software. Artech House, Boston,
London, 1993.
[6] G.V. Puskorius and L.A Feldkamp, " Neuro Control of nonlinear
dynamical systems with Kalman Filter trained recurrent networks"
IEEE Transactions on neural networks, Vol 5, No.2, pp 279-297,
1994.
[7] K.S. Narendra and K Parthasarathy, "Identification and control of
Dynamical systems using neural networks", IEEE Transactions on
Neural Networks, Vol 1, No.2, pp 4-27, March 1990
[8] Shuhi Li, "Comparative Analysis of back propagation and
Extended Kalman filter in Pattern and Batch forms for training
Neural Networks", IEEE Transactions on Neural Network, Vol 2,
No.1, pp144-149, 2001
[9] M.S. Grewal and A.P Andrews, Kalman Filtering Theory and
Practice, Prentice Hall, Englewood Cliffs, 1993.
[10] Y. Linguni, H. Sakai, H. Tokumaru, "A Real Time Learning
Algorithm for Multilayered Neural Network based on the Extended
Kalman Filter", IEEE Transactions on Signal Processing, Vol 40,
No.4, pp 959-966,1992.
[11] Joost H. de Vlieger and Robert H.J. Gmelig Meyling , "Maximum
Likelihood Estimation for Long Range Target Tracking Using
Passive Sonar Measurements", IEEE transactions on Signal
Processing, Vol. 40, No.5, pp 1216-1225,May 1992
[12] Ivan Petrovic, Mato Baotic, Nedjeljko Peric "Model structure
selection for nonlinear system identification using feed forward
neural networks" ,Department of Control and Computer
Engineering in Automation, Unska 3, HR - 10 000 Zagreb,
Croatia.
[13] Simon Haykin, Adaptive Filter Theory, Prentice Hall International
editions, 1986
[14] Anders Forsgren and Robert Kling "Implementation of Recurrent
Neural Networks for Prediction and control of Nonlinear Dynamic
Systems", Lulea University of Technology, Lulea, Sweden
[15] Ben James, Brian D.O, Anderson, and Robert . C. Williamson
"Conditional Mean and Maximum Likelihood approaches to Multi
harmonic frequency estimation.", IEEE Transactions on Signal
Processing, Vol 42, No.6, pp 1366-1375, June 1994.
[16] Langford B White, "Robust Approximate likelihood ratio Tests for
Nonlinear dynamic systems", IEEE Transactions on Signal
Processing, Vol 43, No.8, pp 2028-2031, August 1995.