Evolving Neural Networks using Moment Method for Handwritten Digit Recognition

This paper proposes a neural network weights and topology optimization using genetic evolution and the backpropagation training algorithm. The proposed crossover and mutation operators aims to adapt the networks architectures and weights during the evolution process. Through a specific inheritance procedure, the weights are transmitted from the parents to their offsprings, which allows re-exploitation of the already trained networks and hence the acceleration of the global convergence of the algorithm. In the preprocessing phase, a new feature extraction method is proposed based on Legendre moments with the Maximum entropy principle MEP as a selection criterion. This allows a global search space reduction in the design of the networks. The proposed method has been applied and tested on the well known MNIST database of handwritten digits.

Authors:

• H. El Fadili
• K. Zenkouar
• H. Qjidaa

Keywords:

• Genetic algorithm
• Legendre Moments
• MEP
• Neural Network.

References:

[1] M. K. Hu, "Visual pattern recognition by moment invariants," IRE
Transaction on Information Theory, vol. 8, no. 2, pp. 179-187, 1962.
[2] S. X. Liao and Miroslaw Pawlak, "On image analysis by moments,"
IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 18, no.
3, pp. 254-266, 1996.
[3] H. Qjidaa and L. Redouane, "Robust line fitting in a noisy image by the
method of moments," IEEE Trans. on Pattern Analysis and Machine
Intelligence, vol. 21, no. 11, pp. 1216-1223, 1999.
[4] W. H. schiffmann and K. Mecklenburg, "Genetic Generation of
Backpropagation Trained Neural Networks," Proc. of Parallel
Processing in Neural Systems and Computers(ICNC), Eckmiller R. et al.
(Eds.) pp. 205-208, Elsevier, 1990.
[5] G. Miller P. M. Todd and S. U. Hegde, Designing Neural Networks
using Genetic Algorithms, Proc. Of the third Intern. Conference on
Genetic Algorithms (ICGA), San Mateo (CA), 1989, pp. 379-384.
[6] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. "Gradient-based
learning applied to document recognition," Proceedings of the IEEE,
vol. 86, no.11, pp. 2278-2324, November 1998.
[7] H. El Fadili, K. Zenkouar and H. Qjidaa, "Lapped Block Image Analysis
Via the Method of Legendre Moments," EURASIP Journal on Applied
Signal Processing, vol. 2003, no.9, pp. 902-913, August 2003.
[8] X. Zhunang, R. M. Haralick, and Y. Zhao, "Maximum entropy image
reconstruction," IEEE Trans. Signal Processing, vol. 39, no. 6, pp.
1478-1480, 1991.
[9] D. Parisi, A.Cangelosi and S. Nolfi, "cell division and migration in a
genotype for neural networks," Network: computation in neural systems,
vol. 5, no. 4, 1994.
[10] D. Whitley, T. Starkweather, and C. Bogart, "Genetic algorithms and
neural networks: optimizing connections and connectivity," Parallel
Computing,vol. 14, pp. 347-361, 1990.
[11] Y. Liu and X. Yao (1996), "A population-based learning algorithm
which learns both architectures and weights of neural networks,"
Chinese Journal of Advanced Software Research (Allerton Press, Inc.,
New York, NY 10011), vol. 3, no. 1, pp. 54-65, 1996.