A New Self-Adaptive EP Approach for ANN Weights Training

Evolutionary Programming (EP) represents a methodology of Evolutionary Algorithms (EA) in which mutation is considered as a main reproduction operator. This paper presents a novel EP approach for Artificial Neural Networks (ANN) learning. The proposed strategy consists of two components: the self-adaptive, which contains phenotype information and the dynamic, which is described by genotype. Self-adaptation is achieved by the addition of a value, called the network weight, which depends on a total number of hidden layers and an average number of neurons in hidden layers. The dynamic component changes its value depending on the fitness of a chromosome, exposed to mutation. Thus, the mutation step size is controlled by two components, encapsulated in the algorithm, which adjust it according to the characteristics of a predefined ANN architecture and the fitness of a particular chromosome. The comparative analysis of the proposed approach and the classical EP (Gaussian mutation) showed, that that the significant acceleration of the evolution process is achieved by using both phenotype and genotype information in the mutation strategy.

Authors:

• Kristina Davoian
• Wolfram-M. Lippe

Keywords:

• Artificial Neural Networks (ANN)
• Learning Theory
• Evolutionary Programming (EP)
• Mutation
• Self-Adaptation.

References:

[1] X. Yao, "Evolutionary artificial neural networks", in Encyclopedia of
Computer Science and Technology, Vol. 33, New York: Marcel Dekker,
pp. 137-170, 1995
[2] X. Yao, "Evolving Artificial Neural Networks", in Proc. of the IEEE, 87
(9), pp. 1423-1447, 1999
[3] D. B. Fogel, "Evolving Neural Networks: Selected Medical Applications
and the Effects of Variation Operators", Modeling and Simulation:
Theory and Practice - A Memorial Volume for Professor Walter J.
Karplus, Kluwer Academic Press, Boston, MA, pp. 217-248, 2003
[4] D. G. Landavazo and G. B. Fogel, "Evolved Neural Networks for
Quantitative Structure-Activity Relationships of Anti-HIV Compounds",
in Proc. of the IEEE Congress on Evolutionary Computation, Vol. 1,
Honolulu, HI, USA, pp. 199-204, 2002
[5] A. Abraham, "Meta-Learning Evolutionary Artificial Neural Networks",
Neurocomputing Journal, Elsevier Science, Netherlands, Vol. 56c, pp.
1-38, 2004
[6] A. E. Eiben, R. Hinterding, Z. Michalewicz, "Parameter Control in
Evolutionary Algorithms", IEEE Trans. on Evolutionary Computation,
Vol. 3, pp. 124-141, 2000
[7] R. Hinterding, "Gaussian mutation and self-adaption for numeric genetic
algorithms", in Proc. of the Second IEEE Conference on Evolutionary
Computation, pp. 384-389, 1995
[8] A. Jain, D. Fogel. "Case studies in applying fitness distributions in
evolutionary algorithms: I. Simple neural networks and Gaussian
mutation", Applications and Science of Computational Intelligence III,
Proc. SPIE, Vol. 4055, pp. 168-175, 2000
[9] P. A. Castillo, J. J. Merelo, V. Rivas, G. Romero, and A. Prieto,
"Evolving Multilayer Perceptrons", Neural Processing Letters 12(2),
pp.115-127, 2000
[10] J. Branke, "Evolutionary approaches to dynamic optimization problems
- a survey", GECCO Workshop on Evolutionary Algorithms for
Dynamic Optimization Problems, pp. 134-137, 1999
[11] D. B. Fogel and K. Chellapilla, "Revisiting evolutionary programming",
in SPIE AeroSense'98, Applications and Science of Computational
Intelligence, Orlando, FL, pp. 2-11, 1998
[12] W.-M. Lippe, "Soft-Computing mit Neuronalen Netzen, Fuzzy-Logic
und Evolutionären Algorithmen", Springer-Verlag, Berlin Heidelberg,
2006
[13] H. Abbass and R. Sarker, "Simultaneous evolution of architectures and
connection weights in anns", in Artificial Neural Networks and Expert
Systems Conference, Dunedin, New Zealand, pp. 16-21, 2001
[14] Lock and C. Giraud-Carrier. "Evolutionary Programming of Near-
Optimal Neural Networks", in Proc. of the Fourth International
Conference on Artificial Neural Networks and Genetic Algorithms
(ICANNGA99), Springer-Verlag, pp. 302-306, 1999
[15] X. Yao and Y. Liu, "Fast Evolutionary Programming", in Proc. of the
Fifth Annual Conference on Evolutionary Programming (EP'96), the
MIT Press, San Diego, CA, USA, 29/2-2/3/96. pp. 451-460, 1996
[16] X. Yao and Y. Liu, "Fast evolution strategies," Control and Cybernetics,
26(3), pp. 467-496, 1997
[17] X. Yao, Y. Liu, and G. Lin, "Evolutionary programming made faster",
IEEE Transactions on Evolutionary Computation, pp. 82-102, 1999
[18] K. Davoian, A.Reichel, W.-M. Lippe, "Comparison and analysis of
mutation-based evolutionary algorithms for ANN parameters
optimization", in Proc. of the 2006 International conference on Data
Mining (DMIN-06),CSREA Press, 2006
[19] X. Yao, Y. Liu, "A new evolutionary system for evolving artificial
neural networks", IEEE Transactions on Neural Networks, 8(3): 694-
713, May 1997
[20] Y. Chen, B. Yang, J. Dong, A. Abraham, "Time series forecasting using
flexible neural tree model", Information Sciences: an International
Journal, Vol 174, pp. 219-235, 2005
[21] W. Greiner, L. Neise, H. Stöcker, "Thermodynamics and statistical
mechanics", Springer-Verlag, New York [u.a.] 2000
[22] I. Rechenberg, "Evolutionsstrategie: Optimierung technischer Systeme
nach Prinzipien der biologischen Evolution", Fromman-Holzboog
Verlag, Stuttgart, Germany, 1973
[23] M. Mackey and L. Glass, "Oscillation and chaos in physiological control
systems", Sci., vol. 197, p. 287, 1977