Performance Evaluation of Complex Valued Neural Networks Using Various Error Functions

The backpropagation algorithm in general employs quadratic error function. In fact, most of the problems that involve minimization employ the Quadratic error function. With alternative error functions the performance of the optimization scheme can be improved. The new error functions help in suppressing the ill-effects of the outliers and have shown good performance to noise. In this paper we have tried to evaluate and compare the relative performance of complex valued neural network using different error functions. During first simulation for complex XOR gate it is observed that some error functions like Absolute error, Cauchy error function can replace Quadratic error function. In the second simulation it is observed that for some error functions the performance of the complex valued neural network depends on the architecture of the network whereas with few other error functions convergence speed of the network is independent of architecture of the neural network.

Authors:

• Anita S. Gangal
• P. K. Kalra
• D. S. Chauhan

Keywords:

• Complex backpropagation algorithm
• complex errorfunctions
• complex valued neural network
• split activation function.

References:

[1] B. Widrow, J. McCool, and M. Ball, "The Complex LMS algorithm,"
Proc. of the IEEE, April 1975.
[2] M. S. Kim and C. C. Guest, "Modification of backpropagation for
complex- valued signal processing in frequency domain," Int. Joint
Conf. on Neural Networks, San Diego, vol. 3, pp. 27-31, June 1990.
[3] G. M. Georgiou and C. Koutsougeras, "Complex domain
backpropagation," IEEE Trans. On Circuits and Systems - II: Analog
and Digital Signal Processing, Vol.39, No. 5., May 1992.
[4] A. Hirose, "Dynamics of fully complex-valued neural networks,"
Electronics letters, vol. 28, no. 13, pp.1492-1494.
[5] N. Benvenuoto and F. Piazza, "On the complex backpropagation
algorithm," IEEE Trans. On Signal Processing, vol. 40, no. 4, April
1992.
[6] J. Wang, "Recurrent neural networks for solving Systems of complex
valued linear equations," Electronics letters, vol. 28, no. 18, pp. 1751-
1753.
[7] Y. Deville, "A neural network implementation of complex activation
function for digital VLSI neural networks," Microelectronics Jjournal,
vol. 24, pp. 259-262.
[8] M. R. Smith and Y. Hui, "A data exploration algorithm using a complex
domain neural network," IEEE Trans.on Circuits and systems-II: Analog
and digital signal processing, vol. 22,no.2.
[9] H. Leung and S. Haykin," The complex backpropagation algorithm",
IEEE Trans. On signal Processing, Vol. 39,No.9, September(1991).
[10] T. Nitta, "An extension of the back-propagation algorithm to complex
numbers," Neural Networks, vol. 10, no. 8, pp 1391-1415, 1997.
[11] P. J. Werbos, andJ Titus, "An empirical test of new forecasting methods
derived from a theory of intelligence: The prediction of conflict in Latin
America," IEEE Trans.on Systems, Man and Cybernetics, September,
1978.
[12] P. E. Gill, and M. H. Wright, "Practical optimization," Academic Press,
1981.
[13] W. J. J. Rey, "Introduction to robust and quasi-robust statistical
methods," Springer- Verlag, Berlin.
[14] B. Fernandez, "Tools for artificial neural networks learning," Intelligent
engineering systems through Artificial Neural Networks, ASME Press,
New York, pp. 69-76, 1991.
[15] K. Matsuoka, and J. Yi, "Backpropagation based on the Logarithmic
error function and elimination of local minima," in Proc. Of the
International Joint Conference on Neural Networks, Singapore, vol. 2,
pp. 1117-1122, 1991.
[16] V. Ooyen, Nienhaus, "Improving the convergence of backpropagetion
algorithm," Neural Networks, vol. 5, pp. 465-571.
[17] A. Prashanth, "Investigation on complex variable based backpropagation
algorithm and applications," Ph.D. Thesis, IIT, Kanpur, India, March
2003.