A Probabilistic Reinforcement-Based Approach to Conceptualization

Conceptualization strengthens intelligent systems in generalization skill, effective knowledge representation, real-time inference, and managing uncertain and indefinite situations in addition to facilitating knowledge communication for learning agents situated in real world. Concept learning introduces a way of abstraction by which the continuous state is formed as entities called concepts which are connected to the action space and thus, they illustrate somehow the complex action space. Of computational concept learning approaches, action-based conceptualization is favored because of its simplicity and mirror neuron foundations in neuroscience. In this paper, a new biologically inspired concept learning approach based on the probabilistic framework is proposed. This approach exploits and extends the mirror neuron-s role in conceptualization for a reinforcement learning agent in nondeterministic environments. In the proposed method, instead of building a huge numerical knowledge, the concepts are learnt gradually from rewards through interaction with the environment. Moreover the probabilistic formation of the concepts is employed to deal with uncertain and dynamic nature of real problems in addition to the ability of generalization. These characteristics as a whole distinguish the proposed learning algorithm from both a pure classification algorithm and typical reinforcement learning. Simulation results show advantages of the proposed framework in terms of convergence speed as well as generalization and asymptotic behavior because of utilizing both success and failures attempts through received rewards. Experimental results, on the other hand, show the applicability and effectiveness of the proposed method in continuous and noisy environments for a real robotic task such as maze as well as the benefits of implementing an incremental learning scenario in artificial agents.

• Hadi Firouzi
• Majid Nili Ahmadabadi
• Babak N. Araabi

• Concept learning
• probabilistic decision making
• reinforcement learning.

[1] S. Amizadeh, M. N. Ahmadabadi, B. N. Araabi, R. Siegwart, A Bayesian
Approach to Conceptualization Using Reinforcement Learning,
IEEE/ASME International Conference on Advanced Intelligent
Mechatronics, 2007.
[2] T. R. Zentall, M. Galizio, and T. S. Critchfield, "Categorization, concept
Learning and behavior analysis," in Journal of the Experimental
Analysis of Behavior, vol. 78, no. 3, pp. 237-248, November 2002.
[3] G. Buccino, S. Vogt, A. Ritzl, G. R. Fink, K. Zilles, H. J. Freund and G.
Rizzolatti, "Neural circuits underlying imitation learning of hand
actions: an event related fMRI study," in Neuron, vol. 42, pp. 323-334,
April 2004.
[4] A. Billard and M. J. Mataric, "Automatic learning human arm
movements by imitation: evaluation of a biologically inspired
connectionist architecture," in Robotics and Autonomous Systems, vol.
941, pp. 1-16, 2001.
[5] K. Doya, "Reinforcement learning in continuous time and space,"
Neural Computation, vol. 12, pp. 219-245, 2000.
[6] H. Mobahi, M. Nili Ahmadabadi, and B. N. Araabi, "Concept oriented
imitation towards verbal human-robot interaction," In Proc. 2005 IEEE
Int. Conf. Robotics and Automation, pp. 1495-1500, April 2005.
[7] H. Mobahi, M. Nili Ahmadabadi, and B. N. Araabi, "A biologically
inspired for conceptual imitation using reinforcement learning," to be
published in Applied Artificial Intelligence.
[8] S. Mahadevan and J. Connell, "Automatic programming of behaviorbased
robots using reinforcement learning," in Artificial Intelligence,
vol. 55, no. 2-3, pp. 311-365, June 1992.
[9] A. J. Smith, "Applications of the self-organizing map to reinforcement
learning," in Neural Networks, vol. 15, pp. 1107-1124, 2002.
[10] O. Lebeltel, P. Bessiere, J. Diard and E. Mazer, "Bayesian robot
programming," in Autonomous Robots, vol. 16, pp. 49-79, 2004.
[11] R. O. Duda, P. E. Hart and D. G. Stork, Pattern Classification (2nd
Edition). New York: Wiley-Interscience, 2000.
[12] R. E. Neapolitan, Learning Bayesian Network. New Jersey: Pearson
Prentice Hall, 2003.
[13] C. E. Priebe, "Adaptive mixtures," in Journal of the American Statistical
Association, vol. 89, no. 427, pp. 796-806, September 1994.
[14] E-puck, EPFL Education Robot, http://www.e-puck.org.