In this paper, a novel associative memory model will be proposed and applied to memory retrievals based on the conventional continuous time model. The conventional model presents memory capacity is very low and retrieval process easily converges to an equilibrium state which is very different from the stored patterns. Genetic Algorithms is well-known with the capability of global optimal search escaping local optimum on progress to reach a global optimum. Based on the well-known idea of Genetic Algorithms, this work proposes a heuristic rule to make a mutation when the state of the network is trapped in a spurious memory. The proposal heuristic associative memory show the stored capacity does not depend on the number of stored patterns and the retrieval ability is up to ~ 1.
[1] Masahiro Nakagawa, Chaos and Fractals in Engineering, World
Scientific, Singapore, 1999.
[2] Mohamad H. Hassoun, Associative Neural Memories: Theory and
Implementation, Oxford University Press, USA 1993.
[3] S. Haykin, Neural Networks: A Comprehensive Foundation, Second
Edition, Macmillan, New York, 1999.
[4] C. M. Marcus and R. M. Westervelt, "Stability and convergence of analog
neural networks with multiple-time-step parallel dynamics", Phys. Rev. A
42, 1990, pp.2410
[5] F. R. Waugh, and R. M. Westervelt, "Associative memory in an analog
iterated-map neural network C. M. Marcus", Phys. Rev. A 41, 1990,
pp.3355
[6] Lee, R.S.T., "A transient-chaotic autoassociative network (TCAN) based
on Lee oscillators", IEEE Trans. Neural Networks, Vol. 15, No. 5, Sep.
2004, pp.1228 - 1243
[7] Yanai, H.-F., and Amari, S.-I., "Auto-associative memory with two-stage
dynamics of nonmonotonic neurons", IEEE Trans. Neural Networks, Vol.
7, No. 4, July 1996, pp.803 - 815
[8] Seiji Miyoshi, and Masato Okada, "Storage Capacity Diverges With
Synaptic Efficiency in an Associative Memory Model With Synaptic
Delay and Pruning", IEEE Trans. Neural Networks, Vol. 15, No. 5, Sep.
2004, pp.1215
[9] Masaharu Adachi and Kazuyuki Aihara, "Associative Dynamics in a
Chaotic Neural Network", J. Neural Networks, Vol. 10, No. 1, Jan.
1997, pp. 83-98
[10] Masahiko Morita, "Associative memory with nonmonotone dynamics", J.
Neural Networks, Vol. 6, No. 1, 1993, pp. 115-126
[11] Masahiro Nakagawa, "Statistical Properties of Chaos Associative
Memory", J. of the Physical Society of Japan, Vol. 71, No. 9, Sep. 2002,
pp. 2316-2325
[12] Masahiro Nakagawa, "A Chaos Associative Memory with a Skew-Tent
Activation Function", J. of the Physical Society of Japan, Vol.69, No.3,
Mar. 2000, pp. 972-980
[13] Masahiro Nakagawa, "Chaos Associative Memory with a Periodic
Activation Function", J. of the Physical Society of Japan, Vol. 67, No. 7,
July 1998, pp. 2281-2293
[14] Masahiro Nakagawa, "Entropy Based Associative Memory", IEICE
Trans. Fundamentals of Electronics, Communications and Computer,
Vol.E89-A, No.4, 2006, pp.895-901
[15] Truong Quang Dang Khoa, Masahiro Nakagawa, Neural Network
Learning based on Chaos, WASET Inter. J. Computer, Information, and
Systems Science, and Engineering, Vol. 1, No. 2, 2007, pp.97-102
[1] Masahiro Nakagawa, Chaos and Fractals in Engineering, World
Scientific, Singapore, 1999.
[2] Mohamad H. Hassoun, Associative Neural Memories: Theory and
Implementation, Oxford University Press, USA 1993.
[3] S. Haykin, Neural Networks: A Comprehensive Foundation, Second
Edition, Macmillan, New York, 1999.
[4] C. M. Marcus and R. M. Westervelt, "Stability and convergence of analog
neural networks with multiple-time-step parallel dynamics", Phys. Rev. A
42, 1990, pp.2410
[5] F. R. Waugh, and R. M. Westervelt, "Associative memory in an analog
iterated-map neural network C. M. Marcus", Phys. Rev. A 41, 1990,
pp.3355
[6] Lee, R.S.T., "A transient-chaotic autoassociative network (TCAN) based
on Lee oscillators", IEEE Trans. Neural Networks, Vol. 15, No. 5, Sep.
2004, pp.1228 - 1243
[7] Yanai, H.-F., and Amari, S.-I., "Auto-associative memory with two-stage
dynamics of nonmonotonic neurons", IEEE Trans. Neural Networks, Vol.
7, No. 4, July 1996, pp.803 - 815
[8] Seiji Miyoshi, and Masato Okada, "Storage Capacity Diverges With
Synaptic Efficiency in an Associative Memory Model With Synaptic
Delay and Pruning", IEEE Trans. Neural Networks, Vol. 15, No. 5, Sep.
2004, pp.1215
[9] Masaharu Adachi and Kazuyuki Aihara, "Associative Dynamics in a
Chaotic Neural Network", J. Neural Networks, Vol. 10, No. 1, Jan.
1997, pp. 83-98
[10] Masahiko Morita, "Associative memory with nonmonotone dynamics", J.
Neural Networks, Vol. 6, No. 1, 1993, pp. 115-126
[11] Masahiro Nakagawa, "Statistical Properties of Chaos Associative
Memory", J. of the Physical Society of Japan, Vol. 71, No. 9, Sep. 2002,
pp. 2316-2325
[12] Masahiro Nakagawa, "A Chaos Associative Memory with a Skew-Tent
Activation Function", J. of the Physical Society of Japan, Vol.69, No.3,
Mar. 2000, pp. 972-980
[13] Masahiro Nakagawa, "Chaos Associative Memory with a Periodic
Activation Function", J. of the Physical Society of Japan, Vol. 67, No. 7,
July 1998, pp. 2281-2293
[14] Masahiro Nakagawa, "Entropy Based Associative Memory", IEICE
Trans. Fundamentals of Electronics, Communications and Computer,
Vol.E89-A, No.4, 2006, pp.895-901
[15] Truong Quang Dang Khoa, Masahiro Nakagawa, Neural Network
Learning based on Chaos, WASET Inter. J. Computer, Information, and
Systems Science, and Engineering, Vol. 1, No. 2, 2007, pp.97-102
@article{"International Journal of Information, Control and Computer Sciences:53519", author = "Truong Quang Dang Khoa and Masahiro Nakagawa", title = "Heuristic Continuous-time Associative Memories ", abstract = "In this paper, a novel associative memory model will be proposed and applied to memory retrievals based on the conventional continuous time model. The conventional model presents memory capacity is very low and retrieval process easily converges to an equilibrium state which is very different from the stored patterns. Genetic Algorithms is well-known with the capability of global optimal search escaping local optimum on progress to reach a global optimum. Based on the well-known idea of Genetic Algorithms, this work proposes a heuristic rule to make a mutation when the state of the network is trapped in a spurious memory. The proposal heuristic associative memory show the stored capacity does not depend on the number of stored patterns and the retrieval ability is up to ~ 1. ", keywords = "Artificial Intelligent, Soft Computing, NeuralNetworks, Genetic Algorithms, Hopfield Neural Networks, andAssociative Memories.", volume = "1", number = "3", pages = "543-7", }
