Evolution of Fuzzy Neural Networks Using an Evolution Strategy with Fuzzy Genotype Values
Evolution strategy (ES) is a well-known instance of evolutionary algorithms, and there have been many studies on ES. In this paper, the author proposes an extended ES for solving fuzzy-valued optimization problems. In the proposed ES, genotype values are not real numbers but fuzzy numbers. Evolutionary processes in the ES are extended so that it can handle genotype instances with fuzzy numbers. In this study, the proposed method is experimentally applied to the evolution of neural networks with fuzzy weights and biases. Results reveal that fuzzy neural networks evolved using the proposed ES with fuzzy genotype values can model hidden target fuzzy functions even though no training data are explicitly provided. Next, the proposed method is evaluated in terms of variations in specifying fuzzy numbers as genotype values. One of the mostly adopted fuzzy numbers is a symmetric triangular one that can be specified by its lower and upper bounds (LU) or its center and width (CW). Experimental results revealed that the LU model contributed better to the fuzzy ES than the CW model, which indicates that the LU model should be adopted in future applications of the proposed method.
[1] H. Ishibuchi, H. Tanaka, and H. Okada, Fuzzy neural networks with fuzzy
weights and fuzzy biases, Proc. of IEEE International Conferences on
Neural Networks, pp.1650-1655, 1993.
[2] D.B. Fogel, L.J. Fogel, and V.W. Porto, Evolving neural networks,
Biological Cybernetics, vol.63, issue 6, pp.487-493, 1990.
[3] X. Yao, Evolving artificial neural networks, Proc. of the IEEE, vol.87,
no.9, pp.1423-1447, 1999.
[4] K.O. Stanley and R. Miikkulainen, Evolving neural networks
through augmenting topologies, Evolutionary Computation, vol.10, no.2,
pp.99-127, 2002.
[5] D. Floreano, P. Durr, and C. Mattiussi, Neuroevolution: From
architectures to learning, Evolutionary Intelligence, vol.1, no.1, pp.47-62,
2008.
[6] H. Okada, Genetic algorithm with fuzzy genotype values and its
application to neuroevolution, International Journal of Computer,
Information Science and Engineering, vol.8, no.1, pp.1-7, 2014.
[7] H-P. Schwefel, Evolution and Optimum Seeking, Wiley, 1995.
[8] H-G. Beyer and H-P. Schwefel, Evolution strategies - A comprehensive
introduction, Natural Computing, vol.1, no.1, pp.3-52, 2002.
[9] N. Hansen and A. Ostermeier, Adapting arbitrary normal mutation
distributions in evolution strategies: The covariance matrix adaptation,
Proc. of the 1996 IEEE International Conference on Evolutionary
Computation, pp.312-317, 1996.
[10] M. Herdy, Evolution strategies with subjective selection, Proc. of the
4th International Conference on Parallel Problem Solving from Nature,
pp.22-31, 1996.
[11] J. Knowles and D. Corne, The pareto archived evolution strategy: A
new baseline algorithm for pareto multiobjective optimisation, Proc. of
the 1999 Congress on Evolutionary Computation, pp.98-105, 1999.
[12] N. Hansen and A. Ostermeier, Completely derandomized self-adaptation
in evolution strategies, Evolutionary Computation, vol.9, no.2,
pp.159-195, 2001.
[13] H.G. Beyer, The Theory of Evolution Strategies, Springer, 2001.
[14] A. Auger and N. Hansen, A restart CMA evolution strategy with
increasing population size, Proc. of the 2005 IEEE Congress on
Evolutionary Computation, pp.1769-1776, 2005.
[15] O.M. Shir and T. Back, Niching in evolution strategies, Proc. of the
2005 Conference on Genetic and Evolutionary Computation, pp.915-916,
2005.
[16] N. Hansen, The CMA evolution strategy: A comparing review, Towards
a New Evolutionary Computation, Springer, pp.1769-1776, 2006.
[17] D.V. Arnold, Weighted multirecombination evolution strategies,
Theoretical Computer Science - Foundations of Genetic Algorithms,
vol.361, no. 1, pp.18-37, 2006.
[18] X. Chen, X. Liu, and Y. Jia, Combining evolution strategy and gradient
descent method for discriminative learning of bayesian classifiers, Proc. of
the 11th Annual Conference on Genetic and Evolutionary Computation,
pp.507-514, 2009.
[19] D.V. Arnold and A.S. Castellarin, A novel approach to adaptive isolation
in evolution strategies, Proc. of the 11th Annual Conference on Genetic
and Evolutionary Computation, pp.491-498, 2009.
[20] L. Graening, N. Aulig, and M. Olhofer, Towards directed open-ended
search by a novelty guided evolution strategy, Proc. of the 11th
International Conference on Parallel Problem Solving from Nature: Part
II, pp.71-80, 2010.
[21] A. Auger, D. Brockhoff, and N. Hansen, Analyzing the impact of
mirrored sampling and sequential selection in elitist evolution strategies,
Proc. of the 11th Workshop on Foundations of Genetic Algorithms,
pp.127-138, 2011.
[22] A. Auger, D. Brockhoff, and N. Hansen, Mirrored sampling in evolution
strategies with weighted recombination, Proc. of the 13th Annual
Conference on Genetic and Evolutionary Computation, pp.861-868, 2011.
[23] I. Loshchilov, M. Schoenauer, and M. Sebag, Self-adaptive
surrogate-assisted covariance matrix adaptation evolution strategy,
Proc. of the Fourteenth International Conference on Genetic and
Evolutionary Computation Conference, pp.321-328, 2012.
[24] J. Lu, B. Li, and Y. Jin, An evolution strategy assisted by an ensemble of
local gaussian process models. Proc. of the Fifteenth Annual Conference
on Genetic and Evolutionary Computation Conference, pp.447-454, 2013.
[25] R. Li, M.T.M. Emmerich, J. Eggermont, T. Back, M. Schutz, J. Dijkstra,
and J.H.C. Reiber, Mixed integer evolution strategies for parameter
optimization, Evolutionary Computation, vol.21, no.1 pp.29-64, 2013.
[26] L.A. Zadeh, The concept of a linguistic variable and its application
to approximate reasoning - I, II, and III, Information Sciences, vol.8,
pp.199-249, pp.301-357, and vol.9, pp.43-80, 1975.
[27] G. Alefeld and J. Herzberger, Introduction to Interval Computation,
Academic Press, 1983.
[28] H. Okada, J. Tokida, and Y. Fujii, Comparison of evolution strategy,
genetic algorithm and their hybrids on evolving autonomous game
controller agents, International Journal of Science and Engineering
Investigations, 10612-02, vol.1, no. 6, pp.11-16, 2012.
[1] H. Ishibuchi, H. Tanaka, and H. Okada, Fuzzy neural networks with fuzzy
weights and fuzzy biases, Proc. of IEEE International Conferences on
Neural Networks, pp.1650-1655, 1993.
[2] D.B. Fogel, L.J. Fogel, and V.W. Porto, Evolving neural networks,
Biological Cybernetics, vol.63, issue 6, pp.487-493, 1990.
[3] X. Yao, Evolving artificial neural networks, Proc. of the IEEE, vol.87,
no.9, pp.1423-1447, 1999.
[4] K.O. Stanley and R. Miikkulainen, Evolving neural networks
through augmenting topologies, Evolutionary Computation, vol.10, no.2,
pp.99-127, 2002.
[5] D. Floreano, P. Durr, and C. Mattiussi, Neuroevolution: From
architectures to learning, Evolutionary Intelligence, vol.1, no.1, pp.47-62,
2008.
[6] H. Okada, Genetic algorithm with fuzzy genotype values and its
application to neuroevolution, International Journal of Computer,
Information Science and Engineering, vol.8, no.1, pp.1-7, 2014.
[7] H-P. Schwefel, Evolution and Optimum Seeking, Wiley, 1995.
[8] H-G. Beyer and H-P. Schwefel, Evolution strategies - A comprehensive
introduction, Natural Computing, vol.1, no.1, pp.3-52, 2002.
[9] N. Hansen and A. Ostermeier, Adapting arbitrary normal mutation
distributions in evolution strategies: The covariance matrix adaptation,
Proc. of the 1996 IEEE International Conference on Evolutionary
Computation, pp.312-317, 1996.
[10] M. Herdy, Evolution strategies with subjective selection, Proc. of the
4th International Conference on Parallel Problem Solving from Nature,
pp.22-31, 1996.
[11] J. Knowles and D. Corne, The pareto archived evolution strategy: A
new baseline algorithm for pareto multiobjective optimisation, Proc. of
the 1999 Congress on Evolutionary Computation, pp.98-105, 1999.
[12] N. Hansen and A. Ostermeier, Completely derandomized self-adaptation
in evolution strategies, Evolutionary Computation, vol.9, no.2,
pp.159-195, 2001.
[13] H.G. Beyer, The Theory of Evolution Strategies, Springer, 2001.
[14] A. Auger and N. Hansen, A restart CMA evolution strategy with
increasing population size, Proc. of the 2005 IEEE Congress on
Evolutionary Computation, pp.1769-1776, 2005.
[15] O.M. Shir and T. Back, Niching in evolution strategies, Proc. of the
2005 Conference on Genetic and Evolutionary Computation, pp.915-916,
2005.
[16] N. Hansen, The CMA evolution strategy: A comparing review, Towards
a New Evolutionary Computation, Springer, pp.1769-1776, 2006.
[17] D.V. Arnold, Weighted multirecombination evolution strategies,
Theoretical Computer Science - Foundations of Genetic Algorithms,
vol.361, no. 1, pp.18-37, 2006.
[18] X. Chen, X. Liu, and Y. Jia, Combining evolution strategy and gradient
descent method for discriminative learning of bayesian classifiers, Proc. of
the 11th Annual Conference on Genetic and Evolutionary Computation,
pp.507-514, 2009.
[19] D.V. Arnold and A.S. Castellarin, A novel approach to adaptive isolation
in evolution strategies, Proc. of the 11th Annual Conference on Genetic
and Evolutionary Computation, pp.491-498, 2009.
[20] L. Graening, N. Aulig, and M. Olhofer, Towards directed open-ended
search by a novelty guided evolution strategy, Proc. of the 11th
International Conference on Parallel Problem Solving from Nature: Part
II, pp.71-80, 2010.
[21] A. Auger, D. Brockhoff, and N. Hansen, Analyzing the impact of
mirrored sampling and sequential selection in elitist evolution strategies,
Proc. of the 11th Workshop on Foundations of Genetic Algorithms,
pp.127-138, 2011.
[22] A. Auger, D. Brockhoff, and N. Hansen, Mirrored sampling in evolution
strategies with weighted recombination, Proc. of the 13th Annual
Conference on Genetic and Evolutionary Computation, pp.861-868, 2011.
[23] I. Loshchilov, M. Schoenauer, and M. Sebag, Self-adaptive
surrogate-assisted covariance matrix adaptation evolution strategy,
Proc. of the Fourteenth International Conference on Genetic and
Evolutionary Computation Conference, pp.321-328, 2012.
[24] J. Lu, B. Li, and Y. Jin, An evolution strategy assisted by an ensemble of
local gaussian process models. Proc. of the Fifteenth Annual Conference
on Genetic and Evolutionary Computation Conference, pp.447-454, 2013.
[25] R. Li, M.T.M. Emmerich, J. Eggermont, T. Back, M. Schutz, J. Dijkstra,
and J.H.C. Reiber, Mixed integer evolution strategies for parameter
optimization, Evolutionary Computation, vol.21, no.1 pp.29-64, 2013.
[26] L.A. Zadeh, The concept of a linguistic variable and its application
to approximate reasoning - I, II, and III, Information Sciences, vol.8,
pp.199-249, pp.301-357, and vol.9, pp.43-80, 1975.
[27] G. Alefeld and J. Herzberger, Introduction to Interval Computation,
Academic Press, 1983.
[28] H. Okada, J. Tokida, and Y. Fujii, Comparison of evolution strategy,
genetic algorithm and their hybrids on evolving autonomous game
controller agents, International Journal of Science and Engineering
Investigations, 10612-02, vol.1, no. 6, pp.11-16, 2012.
@article{"International Journal of Information, Control and Computer Sciences:71029", author = "Hidehiko Okada", title = "Evolution of Fuzzy Neural Networks Using an Evolution Strategy with Fuzzy Genotype Values", abstract = "Evolution strategy (ES) is a well-known instance of evolutionary algorithms, and there have been many studies on ES. In this paper, the author proposes an extended ES for solving fuzzy-valued optimization problems. In the proposed ES, genotype values are not real numbers but fuzzy numbers. Evolutionary processes in the ES are extended so that it can handle genotype instances with fuzzy numbers. In this study, the proposed method is experimentally applied to the evolution of neural networks with fuzzy weights and biases. Results reveal that fuzzy neural networks evolved using the proposed ES with fuzzy genotype values can model hidden target fuzzy functions even though no training data are explicitly provided. Next, the proposed method is evaluated in terms of variations in specifying fuzzy numbers as genotype values. One of the mostly adopted fuzzy numbers is a symmetric triangular one that can be specified by its lower and upper bounds (LU) or its center and width (CW). Experimental results revealed that the LU model contributed better to the fuzzy ES than the CW model, which indicates that the LU model should be adopted in future applications of the proposed method.
", keywords = "Evolutionary algorithm, evolution strategy, fuzzy number, feedforward neural network, neuroevolution.", volume = "9", number = "4", pages = "1029-8", }
