A Probabilistic View of the Spatial Pooler in Hierarchical Temporal Memory

In the Hierarchical Temporal Memory (HTM) paradigm the effect of overlap between inputs on the activation of columns in the spatial pooler is studied. Numerical results suggest that similar inputs are represented by similar sets of columns and dissimilar inputs are represented by dissimilar sets of columns. It is shown that the spatial pooler produces these results under certain conditions for the connectivity and proximal thresholds. Following the discussion of the initialization of parameters for the thresholds, corresponding qualitative arguments about the learning dynamics of the spatial pooler are discussed.

Authors:

• Mackenzie Leake
• Liyu Xia
• Kamil Rocki
• Wayne Imaino

Keywords:

• Hierarchical Temporal Memory
• HTM
• Learning Algorithms
• Machine Learning
• Spatial Pooler.

References:

[1] B. Bobier and M.Wirth, “Content-based image retrieval using hierarchical
temporal memory,” in Proc. 16th ACM Int. Conf. on Multimedia, 2008,
pp. 925-928.
[2] P. Gabrielsson, R. Konig, and U. Johansson, “Evolving hierarchical
temporal memory-based trading models,” in EvoApplications
2013-Applications of Evolutionary Computing, Vienna, April 3-5,
2013.
[3] J. Hawkins, S. Ahmad, and D. Dubinsky, “Hierarchical temporal memory
including HTM cortical learning algorithms,” Numenta, Redwood City,
CA, Tech. Rep. ver. 0.2.1, 2011.
[4] D.O. Hebb, ”The first stage of perception: growth of the assembly,” in
The Organization of Behavior, New York, Wiley, 1949, intro. and ch. 4,
pp. xi-xix, 60-78.
[5] D. Maltoni, “Pattern recognition by hierarchical temporal memory,” DEIS
Univ. Bologna, Tech. Rep., pp. 1-46, Apr. 13, 2011.
[6] V. Mountcastle, “The columnar organization of the neocortex,” Brain, vol.
120(4), pp. 701-722, 1997.
[7] A.J. Perea, J.E. Merono, and M.J. Aguilera, “Application of Numenta
hierarchical temporal memory for land-use classification,” S. Afr. J. Sci.,
vol. 105, pp. 370-375, Sept./Oct. 2009.
[8] J. Thornton and A. Srbic, “Spatial pooling for greyscale images,” Int. J.
Mach. Learn. & Cyber., vol. 4, pp. 207-216, 2013.
[9] J. van Doremalen and L. Boves, “Spoken digit recognition using a
hierarchical temporal memory,” Interspeech, pp. 2566-2569, Brisbane,
Sept. 22-26, 2008.