Self-evolving Neural Networks Based On PSO and JPSO Algorithms

A self-evolution algorithm for optimizing neural networks using a combination of PSO and JPSO is proposed. The algorithm optimizes both the network topology and parameters simultaneously with the aim of achieving desired accuracy with less complicated networks. The performance of the proposed approach is compared with conventional back-propagation networks using several synthetic functions, with better results in the case of the former. The proposed algorithm is also implemented on slope stability problem to estimate the critical factor of safety. Based on the results obtained, the proposed self evolving network produced a better estimate of critical safety factor in comparison to conventional BPN network.

• Abdussamad Ismail
• Dong-Sheng Jeng

• Neural networks
• Topology evolution
• Particle swarm optimization.

[1] Miller, G. F., Todd, P. M. and Hegde, S. U. (1989). "Designing neural
networks using genetic algorithms." Proc. 3rd Int. Conf. Genetic
Algorithms and Their Applications, J. D. Schaffer, Ed. San Mateo, CA:
Morgan Kaufmann, pp. 379-384.
[2] Angeline, P. J., Saunders, G. M. and Pollack, J. B. (1994). " An
evolutionary algorithm that constructs recurrent neural networks, IEEE
Transactions on Neural Networks, 5, 54/65
[3] Yao, X. and Liu, Y. (1997). "A New Evolutionary System for Evolving
Artificial Neural Networks." IEEE Transactions on Neural Networks, 8-
3:694-713
[4] Stanley, K. and Miikkulainen, R. (2002). "Evolving Neural Networks
through Augmenting Topologies." Evolutionary Computation, 10(2): 99-
127
[5] Xian-Lun T., Yon-Guo L. and Ling Z. (2007). "A hybrid particle swarm
algorithm for the structure and parameter optimization of feedforward
neural networks. " LNCS 4493:213-218.
[6] Yu, J., Wang, S. and Xi, L. (2008). "Evolving artificial neural networks
using an improved PSO and DPSO." Neurocomputing, 71:1054-1060
[7] Kiranyaz, S., Ince, T., Yildirim, A. and Gabbouj, M.
(2009)."Evolutionary artificial neural networks by multi-dimensional
particle swarm optimization." Neural Networks (2009) in press.
[8] Mat`─▒nez Garc`─▒a ,F. J. and Moreno P'erez J. A. (2008) "Jumping Frogs
Optimization: a New Swarm Method for Discrete Optimization." ,
Technical Report DEIOC 3/2008, Department of Statistics, O.R. and
Computing, University of La Laguna, Tenerife, Spain
[9] Kennedy, J. and Eberhart, R. C. (1997). "A Discrete Binary Version of
the Particle Swarm Algorithm." Proceedings of IEEE Conference on
Systems, Man, and Cybernetics, iscataway, New Jersey,USA. 4104-
4109.
[10] Sopena, J.M., Romero, E. and Alquezar, R. (1999); "Neural networks
with periodic and monotonic activation functions: a comparative study in
classification problems." Ninth International Conference on Artificial
Neural Networks (ICANN ÔÇÿ99). 1:323 - 328.
[11] Wong, K.-W., Leung, C.S. and Chang, S.-J. (2002). "Use of periodic and
monotonic activation functions in multilayer feedforward neural
networks trained by extended Kalman filter algorithm." IEEE
Proceedings. Image Signal Processing, 149 (4), 217 - 224.
[12] Jin, Y., Okabe, T. and Sendhoff, B. (2004). "Neural network
regularization and ensembling using multi-objective evolutionary
algorithms." Congress on Evolutionary Computation (CEC-04), IEEE
[13] Reed , R.D. and Marks, R.J. (1999). Neural Smithing. The MIT Press
[14] Narendra, K. S. and Li, S.-M.(1996). Neural networks in control
systems. In P. Smolensky,M. C. Mozer, and D. E. Rumelhart, editors,
Mathematical Perspectives on Neural Networks, chapter 11, pages 347-
394. Lawrence Erlbaum Associates
[15] Bishop, A .W. (1955). "The use of the slip circle in the stability analysis
of slopes." Geotechnique, 5: 7-17.
[16] Janbu, N.(1973). Slope Stability Computations. Embankment Dam
Engineering - Casagrande Volume, R.C. Hirschfeld and S.J. Poulos,
eds., John Wiley and Sons, New York, pp 47-86.
[17] Morgensternn.,R ., and Price,V .E. (1967). "A numerical method for
solving the equations of stability of general slip surfaces." Computer
Journal, 9: 388-393.