Word Recognition and Learning based on Associative Memories and Hidden Markov Models
A word recognition architecture based on a network
of neural associative memories and hidden Markov models has been
developed. The input stream, composed of subword-units like wordinternal
triphones consisting of diphones and triphones, is provided
to the network of neural associative memories by hidden Markov
models. The word recognition network derives words from this input
stream. The architecture has the ability to handle ambiguities on
subword-unit level and is also able to add new words to the
vocabulary during performance. The architecture is implemented to
perform the word recognition task in a language processing system
for understanding simple command sentences like “bot show apple".
[1] L. Rabiner and B. H. Juang, Fundemantal of Speech Recognition,
Prentice-Hall, Inc., Upper Saddle River,, 1993.
[2] S. Young, et al., The HTK book for HTK version 3.2.1., Cambrige
University, Engineering Department, 2002.
[3] G. Palm, "On associative memory," Biological Cybernetics, vol. 36, pp.
19-31, 1980.
[4] F. Schwenker, F. T. Sommer and G. Palm, "Iterative retrieval of sparsely
coded associative memory patterns," Neural Networks, vol. 9(3), pp.
445-455, 1996.
[5] G. Palm, F. Kurfess, F. Schwenker and A. Strey, Neural Associative
Memories. Technical Report, Universitat Ulm, Germany, 1993.
[6] A. Knoblauch and G. Palm, "Pattern separation and synchronization in
spiking associative memories and visual areas," Neural Networks, vol.
14, pp. 763-780, 2001.
[7] H. Markert, A. Knoblauch and G. Palm, "Modeling of syntactical
processing in the cortex," BioSystems, vol. 89, pp. 300-315, 2007.
[8] D. Willshaw, O. Buneman and H. Longuet-Higgins, "Non-holographic
associative memory," Nature, vol. 222, pp. 960-962, 1969.
[9] G. Palm, "Memory capacities of local rules for synaptic modification. A
comparative review," Concepts in Neuroscience, vol. 2, pp. 97-128,
1991.
[10] J. Buckingham and D. Willshaw, "Performance characteristics of
associative nets," Network:Computation in Neural Systems, vol. 3, pp.
407-414, 1992.
[11] F. T. Sommer and G. Palm, "Improved bidirectional retrieval of sparse
patterns stored by hebbian learning," Neural Networks, vol. 12, pp. 281-
297, 1999.
[12] D. O. Hebb, The Organization of Behaviour, John Wiley, Newyork,
1949.
[13] TIMIT Acoustic-Phonetic Continuous Speech Corpus, National Institute
of Standartsand Technology Speech Dics 1-1.1, NTIS Order No. PB91-
505065, 1990.
[14] Z. Kara Kayikci, H. Markert and G. Palm, "Neural associative memories
and hidden Markov models for speech recognition," presented at 2007
Int. Joint Conf. on Neural Networks, Orlando, Florida (USA).
[15] S. Young, ATK An Application Toolkit for HTK Version 1.6, Machine
Intelligence Laboratory, Cambrige University, Engineering Department,
2007.
[16] Z. Kara Kayikci, D. Zaykovskiy, H. Markert, W. Minker and G. Palm
Distributed Architecture for Speech Controlled Systems Based on
Associative Memories Chapter in Mathematical Analysis of Evolution,
Information and Complexity, Wiley-VCH, Weinheim (Germany),
unpublished.
[1] L. Rabiner and B. H. Juang, Fundemantal of Speech Recognition,
Prentice-Hall, Inc., Upper Saddle River,, 1993.
[2] S. Young, et al., The HTK book for HTK version 3.2.1., Cambrige
University, Engineering Department, 2002.
[3] G. Palm, "On associative memory," Biological Cybernetics, vol. 36, pp.
19-31, 1980.
[4] F. Schwenker, F. T. Sommer and G. Palm, "Iterative retrieval of sparsely
coded associative memory patterns," Neural Networks, vol. 9(3), pp.
445-455, 1996.
[5] G. Palm, F. Kurfess, F. Schwenker and A. Strey, Neural Associative
Memories. Technical Report, Universitat Ulm, Germany, 1993.
[6] A. Knoblauch and G. Palm, "Pattern separation and synchronization in
spiking associative memories and visual areas," Neural Networks, vol.
14, pp. 763-780, 2001.
[7] H. Markert, A. Knoblauch and G. Palm, "Modeling of syntactical
processing in the cortex," BioSystems, vol. 89, pp. 300-315, 2007.
[8] D. Willshaw, O. Buneman and H. Longuet-Higgins, "Non-holographic
associative memory," Nature, vol. 222, pp. 960-962, 1969.
[9] G. Palm, "Memory capacities of local rules for synaptic modification. A
comparative review," Concepts in Neuroscience, vol. 2, pp. 97-128,
1991.
[10] J. Buckingham and D. Willshaw, "Performance characteristics of
associative nets," Network:Computation in Neural Systems, vol. 3, pp.
407-414, 1992.
[11] F. T. Sommer and G. Palm, "Improved bidirectional retrieval of sparse
patterns stored by hebbian learning," Neural Networks, vol. 12, pp. 281-
297, 1999.
[12] D. O. Hebb, The Organization of Behaviour, John Wiley, Newyork,
1949.
[13] TIMIT Acoustic-Phonetic Continuous Speech Corpus, National Institute
of Standartsand Technology Speech Dics 1-1.1, NTIS Order No. PB91-
505065, 1990.
[14] Z. Kara Kayikci, H. Markert and G. Palm, "Neural associative memories
and hidden Markov models for speech recognition," presented at 2007
Int. Joint Conf. on Neural Networks, Orlando, Florida (USA).
[15] S. Young, ATK An Application Toolkit for HTK Version 1.6, Machine
Intelligence Laboratory, Cambrige University, Engineering Department,
2007.
[16] Z. Kara Kayikci, D. Zaykovskiy, H. Markert, W. Minker and G. Palm
Distributed Architecture for Speech Controlled Systems Based on
Associative Memories Chapter in Mathematical Analysis of Evolution,
Information and Complexity, Wiley-VCH, Weinheim (Germany),
unpublished.
@article{"International Journal of Information, Control and Computer Sciences:50835", author = "Zöhre Kara Kayikci and Günther Palm", title = "Word Recognition and Learning based on Associative Memories and Hidden Markov Models", abstract = "A word recognition architecture based on a network
of neural associative memories and hidden Markov models has been
developed. The input stream, composed of subword-units like wordinternal
triphones consisting of diphones and triphones, is provided
to the network of neural associative memories by hidden Markov
models. The word recognition network derives words from this input
stream. The architecture has the ability to handle ambiguities on
subword-unit level and is also able to add new words to the
vocabulary during performance. The architecture is implemented to
perform the word recognition task in a language processing system
for understanding simple command sentences like “bot show apple".", keywords = "Hebbian learning, hidden Markov models, neuralassociative memories, word recognition.", volume = "2", number = "1", pages = "36-5", }
