Towards Growing Self-Organizing Neural Networks with Fixed Dimensionality

The competitive learning is an adaptive process in which the neurons in a neural network gradually become sensitive to different input pattern clusters. The basic idea behind the Kohonen-s Self-Organizing Feature Maps (SOFM) is competitive learning. SOFM can generate mappings from high-dimensional signal spaces to lower dimensional topological structures. The main features of this kind of mappings are topology preserving, feature mappings and probability distribution approximation of input patterns. To overcome some limitations of SOFM, e.g., a fixed number of neural units and a topology of fixed dimensionality, Growing Self-Organizing Neural Network (GSONN) can be used. GSONN can change its topological structure during learning. It grows by learning and shrinks by forgetting. To speed up the training and convergence, a new variant of GSONN, twin growing cell structures (TGCS) is presented here. This paper first gives an introduction to competitive learning, SOFM and its variants. Then, we discuss some GSONN with fixed dimensionality, which include growing cell structures, its variants and the author-s model: TGCS. It is ended with some testing results comparison and conclusions.

Authors:

• Guojian Cheng
• Tianshi Liu
• Jiaxin Han
• Zheng Wang

Keywords:

• Artificial neural networks
• Competitive learning
• Growing cell structures
• Self-organizing feature maps.

References:

[1] B. Fritzke, Some competitive learning methods.
http://www.neuroinformatik..ruhr-uni-bochum.de/ini/VDM/research/gsn/
JavaPaper/
[2] T. Kohonen, Self-Organizing Maps. Springer, Berlin, Heidelberg, 2001.
(Third Extended Edition).
[3] Fritzke, Growing cell structures - a self-organizing network for
unsupervised and supervised learning. Neural Networks,
7(9):1441-1460, 1994.
[4] A. Zell, M. Schmalzl, Dynamic LVQÔÇöA fast neural net learning
algorithm. In Proc. ICANN-94, International Conference on Artificial
Neural Networks, volume II, pages 1095-1098, London, UK, 1994.
[5] J. Goppert, W. Rosenstiel, Interpolation in SOM: Improved
generalization by iterative methods. In Proc. ICANN-95, International
Conference on Artificial Neural Networks, pages 69-74, France, 1995.
[6] A. Zell, H. Bayer, H., Bauknecht, Similarity analysis of molecules with
self-organizing surfacesÔÇöan extension of the self-organizing map. In
Proc. ICNN-94, International Conference on Neural Networks, pages
719-724, Piscataway, NJ, 1994. IEEE Service Center.
[7] D. Deng, N. Kasabov, On-line pattern analysis by evolving
self-organizing maps Neurocomputing 51: 87-103 (2003)
[8] J. Blackmore, Visualizing high-dimensional structure with the
incremental grid growing neural network. Technical Report AI95-238,
University of Texas, Austin, August 1, 1995.
[9] M. Dittenbach, F. Merkl, A. Rauber, The growing hierarchical
self-organizing map. In S. Amari, C. L. Giles, M. Gori, and V. Puri,
editors, Proc of the International Joint Conference on Neural Networks
(IJCNN 2000), volume VI, pages 15 - 19, Como, Italy, July 24. - 27.
2000. IEEE Computer Society.
[10] J. Bruske, G. Sommer, Dynamic cell structure learns perfectly topology
preserving map. Neural Computation, 7(4):845-865, 1995.
[11] T. Martinetz, Competitive Hebbian learning rule forms perfectly
topology preserving maps. In Stan Gielen and Bert Kappen, editors, Proc.
ICANN-93, Int. Conf. on Artificial Neural Networks, pages 427-434,
London, UK, 1993. Springer.
[12] V. Hodge, J. Austin, Hierarchical growing cell structures: TreeGCS. In
IEEE TKDE Special Issue on Connectionist Models for Learning in
Structured Domains.
[13] N. Vlassis, A. Dimopoulos, G. Papakonstantinou, The probabilistic
growing cell structures algorithm. Lecture Notes in Computer Science,
1327:P649, 1997.
[14] J.Pakkanen, J. Iivarinen, E. Oja, The evolving tree -- a novel
self-organizing network for data analysis. Neural Processing Letters,
20(3):199{211, 2004.
[15] Y. Wang, C. Yang, K. Mathee, G. Narasimhan, Clustering using Adaptive
Self-Organizing Maps (ASOM) and Applications. Proceedings of
International Workshop on Bioinformatics Research and Applications,
p944-951 Atlanta, Georgia, May 2005.
[16] R. Freeman and H. Yin, "Adaptive Topological Tree Structure (ATTS)
for document organization and visualization," Neural Networks, Vol. 17,
pp. 1255-1271, 2004.
[17] M. White, S. Fahlman, CMU repository of neural network benchmarks.
Technical report, The Carnegie Mellon University, 1997.