Learning Classifier Systems Approach for Automated Discovery of Crisp and Fuzzy Hierarchical Production Rules

This research presents a system for post processing of data that takes mined flat rules as input and discovers crisp as well as fuzzy hierarchical structures using Learning Classifier System approach. Learning Classifier System (LCS) is basically a machine learning technique that combines evolutionary computing, reinforcement learning, supervised or unsupervised learning and heuristics to produce adaptive systems. A LCS learns by interacting with an environment from which it receives feedback in the form of numerical reward. Learning is achieved by trying to maximize the amount of reward received. Crisp description for a concept usually cannot represent human knowledge completely and practically. In the proposed Learning Classifier System initial population is constructed as a random collection of HPR–trees (related production rules) and crisp / fuzzy hierarchies are evolved. A fuzzy subsumption relation is suggested for the proposed system and based on Subsumption Matrix (SM), a suitable fitness function is proposed. Suitable genetic operators are proposed for the chosen chromosome representation method. For implementing reinforcement a suitable reward and punishment scheme is also proposed. Experimental results are presented to demonstrate the performance of the proposed system.

Authors:

• Suraiya Jabin
• Kamal K. Bharadwaj

Keywords:

• Hierarchical Production Rule
• Data Mining
• Learning Classifier System
• Fuzzy Subsumption Relation
• Subsumption matrix
• Reinforcement Learning.

References:

[1] Bharadwaj, K. K. and R. Varshneya, "Parallelization of Hierarchical Censored production Rules (HCPRs)", Information and Software
Technology, 37, 1995, pp. 453 - 460.
[2] Bharadwaj, K. K. and N.K. Jain, ÔÇÿHierarchical Censored production Rules System-, Data and Knowledge Engineering, North Holland, vol. 8, page 19 - 34, 1992.
[3] Bruha , Ivan, "Pre - and Post - processing in Machine learning and Data mining", ACAI -99, LNAI 2049, pp. 258 - 266, 2001, Copyright -
Verlag Berlin Heidelberg 2001.
[4] Han, J. and Y. FU, "Dynamic generation and refinement of concept
hierarchies for knowledge discovery in databases," AAAI-94 Workshop
Knowledge in Databases (KDD-94), Seattle, WA, 1994, pp. 157-168.
[5] Holland, J.H. (1986a). Escaping brittleness: the possibilities of generalpurpose
learning algorithms applied to parallel rule-based systems. In
Michalski, Carbonell, & Mitchell (eds) Machine learning, an artificial
intelligence approach. Morgan Kaufmann.
[6] B. Liu, M. Hu, and W. Hsu, "Multi-level organization and
summarization of the discovered rules," SIGKDD-2000, Boston, USA,
Aug 20-23, 2000, pp. 208-217.
[7] Holland, J.H. (1975) Adaptation in Natural and Artificial Systems.
University of Michigan Press.
[8] Basheer M. Al-Maqaleh, and K. K. Bharadwaj, "Genetic programming
approach for automated discovery of production rules with fuzzy
hierarchy," in Proc. of National Conference on Methods and Models in
Computing (NCM2C-2006), Jawaharlal Nehru University, New Delhi,
India, 18-19 December 2006, pp.127-134.
[9] Alwyn Barry, John Holmes, and Xavier Llora, ÔÇÿData Mining using
Learning Classifier Systems-, pages 21 - 23
[10] Jabin, Suraiya and K. K. Bharadwaj, "Learning Classifier Systems
Approach for Automated Discovery of Censored Production Rules", In
14. International Enformatika Conference, v14-57, Vol 14 Aug 2006
ISSN 1305 - 5313.
[11] N.J Radcliffe and P.D. Surry, ÔÇÿCo - operation through hierarchical
competition in genetic data mining-, EPCC - TR94 - 09, 1994.
[12] R. A. Angryk, and F. E. Petry, "Mining multi-level associations with
fuzzy hierarchies," The 2005 IEEE International Conference on Fuzzy
Systems, 2005, pp.785-790.
[13] Lanzi, Pier Luca, Wolfgang Stolzmann, and Stewart W. Wilson, editors.
Learning Classifier Systems. From Foundations to Applications, volume
1813 of LNAI, Berlin, 2000. Springer-Verlag.
[14] Suryanto, H. and P. Compton, "Discovery of class relations in exception
structured knowledge bases", ICCS - 2000, Springer, Germany, 2000,
pp. 113 - 126.
[15] John H. Holmes, ÔÇÿDiscovering Risk of Disease with a Learning
Classifier System-, In Thomas Back, editor, Proceedings of the 7th
International Conference on Genetic Algorithms (ICGA97). Morgan
Kaufmann, 1997.
[16] Jain, N. K. and K. K. Bharadwaj, "Some learning techniques in
hierarchical censored production rules (HCPRs) systems" International
Journal of Intelligent Systems, John Wiley & Sons, Inc.(NY),
vol.13,1998, pp.319-344.
[17] Lanzi , P. L. and R. L. Riolo. A roadmap to the last decade of learning
classifier research (from 1989 to 1999). In Lanzi et al. [20], pages 33-
61.
[18] Sutton, R. S. and A. G. Barto. Reinforcement Learning: An Introduction.
MIT Press, 1998.
[19] Bagnall , A. J. Bagnall, G. C. Cawley, in ÔÇÿLearning Classifier Systems
for Data Mining: A Comparison of XCS with Other Classifiers for the
Forest Cover Data Set-.