Rule Insertion Technique for Dynamic Cell Structure Neural Network
This paper discusses the idea of capturing an expert’s knowledge in the form of human understandable rules and then inserting these rules into a dynamic cell structure (DCS) neural network. The DCS is a form of self-organizing map that can be used for many purposes, including classification and prediction. This particular neural network is considered to be a topology preserving network that starts with no pre-structure, but assumes a structure once trained. The DCS has been used in mission and safety-critical applications, including adaptive flight control and health-monitoring in aerial vehicles. The approach is to insert expert knowledge into the DCS before training. Rules are translated into a pre-structure and then training data are presented. This idea has been demonstrated using the well-known Iris data set and it has been shown that inserting the pre-structure results in better accuracy with the same training.
[1] A. S. Garcez, D. M. Gabbay, K. B. Broda, Neural-Symbolic Learning Systems: Foundations and Applications, Springer-Verleg, 2002.
[2] M. Hoehfeld and S. E. Fahlman, “Learning with limited numerical precision using the cascade-correlation algorithm,” IEEE Transactions on Neural Networks, vol. 3, no. 4, pp. 602-611, 1992.
[3] Z. Kurd, T. Kelly, and J. Austin, “Safety criteria and safety lifecycle for artificial neural networks,” In Proceedings of Eunite, vol. 2003, 2003.
[4] G. G Towell and J. W. Shavlik, “Using symbolic learning to improve knowledge-based neural networks,” In AAAI, pp. 177-182, 1992.
[5] G. G Towell and J. W. Shavlik, “Extracting refined rules from knowledge-based neural networks,” In Machine learning, vol. 13, no. 1, pp. 71-101, 1993.
[6] C. L. Giles and C. W. Omlin, “Extraction, insertion and refinement of symbolic rules in dynamically driven recurrent neural networks,” Connection Science, vol. 5, no. 3-4, pp. 307-337, 1993.
[7] M. Charles and C. Jorgensen, Direct adaptive aircraft control using dynamic cell structure neural networks. NASA Technical Memorandum, Ames Research Center, 1997.
[8] M. Darrah, A. Rubenstein, E. Sorton, and B. DeRoos, “On-board health-state awareness to detect degradation in multirotor systems,” In Proceedings of International Conference on Unmanned Aircraft Systems (ICUAS), pp. 1134-1141, 2018.
[9] J. Bruske and G. Sommer, “Dynamic cell structures,” Advances in neural information processing systems, pp. 497-504, 1995.
[10] B. Fritzke, “Growing cell structures a self-organizing network for unsupervised and supervised learning,” Neural networks, vol. 7, no. 9, pp. 1441-1460, 1994.
[11] T. Martinetz, “Competitive hebbian learning rule forms perfectly topology preserving maps,” In ICANN’93, pp. 427-434, 1993.
[12] Darrah, M. and Taylor, Brian. (2011) Chapter 5: Rule Extraction to Understand Changes in an Adaptive System in Adaptive Control Approach for Software Quality Improvement (W. Eric Wong and Bojan Cukic editors) World Scientific. 115-144.
[13] M. Darrah, B. J. Taylor, and S. T. Skias. “Rule extraction from dynamic cell structure neural network used in a safety critical application,” In Proceedings of Florida Artificial Intelligence Research Symposium, Miami, FL, May 2004.
[14] L. L. Pullum, B. J. Taylor, and M. Darrah, Guidance for the Verification and Validation of Neural Networks, vol. 11. John Wiley & Sons, 2007.
[15] B. J. Taylor and M. A. Darrah, “Rule extraction as a formal method for the verification and validation of neural networks.” In Proceedings of 2005 IEEE International Joint Conference on Neural Networks, vol. 5, pp. 2915-2920, 2005.
[16] G. Bologna and Y. Hayashi, “A Comparison Study on Rule Extraction from Neural Network Ensembles, Boosted Shallow Trees, and SVMs,” Applied Computational Intelligence and Soft Computing, vol. 2018, Article ID 4084850, 20 pages, 2018. https://doi.org/10.1155/2018/4084850.
[17] Y. xX. Liu, F. Doctor, S. Z. Fan, and J. S. Shieh, “Performance Analysis of Extracted Rule-Base Multivariable Type-2 Self-Organizing Fuzzy Logic Controller Applied to Anesthesia,” BioMed Research International, vol. 2014, Article ID 379090, 19 pages, 2014. https://doi.org/10.1155/2014/379090.
[18] R. Setiono and H. Liu, “Understanding neural networks via rule extraction,” In IJCAI, vol. 1, pp. 480-485, 1995.
[19] S.M. Kamruzzaman and A. R. Hasan, “Rule extraction using artificial neural networks, arXiv, preprint arXiv:1009.4984, 20102010.
[20] M. Darrah, B. J. Taylor, M. Webb, and R. Livingston. “A geometric rule extraction approach used for verification and validation of a safety critical application,” in Proceedings of Florida Artificial Intelligence Research Symposium Conference, 2005.
[1] A. S. Garcez, D. M. Gabbay, K. B. Broda, Neural-Symbolic Learning Systems: Foundations and Applications, Springer-Verleg, 2002.
[2] M. Hoehfeld and S. E. Fahlman, “Learning with limited numerical precision using the cascade-correlation algorithm,” IEEE Transactions on Neural Networks, vol. 3, no. 4, pp. 602-611, 1992.
[3] Z. Kurd, T. Kelly, and J. Austin, “Safety criteria and safety lifecycle for artificial neural networks,” In Proceedings of Eunite, vol. 2003, 2003.
[4] G. G Towell and J. W. Shavlik, “Using symbolic learning to improve knowledge-based neural networks,” In AAAI, pp. 177-182, 1992.
[5] G. G Towell and J. W. Shavlik, “Extracting refined rules from knowledge-based neural networks,” In Machine learning, vol. 13, no. 1, pp. 71-101, 1993.
[6] C. L. Giles and C. W. Omlin, “Extraction, insertion and refinement of symbolic rules in dynamically driven recurrent neural networks,” Connection Science, vol. 5, no. 3-4, pp. 307-337, 1993.
[7] M. Charles and C. Jorgensen, Direct adaptive aircraft control using dynamic cell structure neural networks. NASA Technical Memorandum, Ames Research Center, 1997.
[8] M. Darrah, A. Rubenstein, E. Sorton, and B. DeRoos, “On-board health-state awareness to detect degradation in multirotor systems,” In Proceedings of International Conference on Unmanned Aircraft Systems (ICUAS), pp. 1134-1141, 2018.
[9] J. Bruske and G. Sommer, “Dynamic cell structures,” Advances in neural information processing systems, pp. 497-504, 1995.
[10] B. Fritzke, “Growing cell structures a self-organizing network for unsupervised and supervised learning,” Neural networks, vol. 7, no. 9, pp. 1441-1460, 1994.
[11] T. Martinetz, “Competitive hebbian learning rule forms perfectly topology preserving maps,” In ICANN’93, pp. 427-434, 1993.
[12] Darrah, M. and Taylor, Brian. (2011) Chapter 5: Rule Extraction to Understand Changes in an Adaptive System in Adaptive Control Approach for Software Quality Improvement (W. Eric Wong and Bojan Cukic editors) World Scientific. 115-144.
[13] M. Darrah, B. J. Taylor, and S. T. Skias. “Rule extraction from dynamic cell structure neural network used in a safety critical application,” In Proceedings of Florida Artificial Intelligence Research Symposium, Miami, FL, May 2004.
[14] L. L. Pullum, B. J. Taylor, and M. Darrah, Guidance for the Verification and Validation of Neural Networks, vol. 11. John Wiley & Sons, 2007.
[15] B. J. Taylor and M. A. Darrah, “Rule extraction as a formal method for the verification and validation of neural networks.” In Proceedings of 2005 IEEE International Joint Conference on Neural Networks, vol. 5, pp. 2915-2920, 2005.
[16] G. Bologna and Y. Hayashi, “A Comparison Study on Rule Extraction from Neural Network Ensembles, Boosted Shallow Trees, and SVMs,” Applied Computational Intelligence and Soft Computing, vol. 2018, Article ID 4084850, 20 pages, 2018. https://doi.org/10.1155/2018/4084850.
[17] Y. xX. Liu, F. Doctor, S. Z. Fan, and J. S. Shieh, “Performance Analysis of Extracted Rule-Base Multivariable Type-2 Self-Organizing Fuzzy Logic Controller Applied to Anesthesia,” BioMed Research International, vol. 2014, Article ID 379090, 19 pages, 2014. https://doi.org/10.1155/2014/379090.
[18] R. Setiono and H. Liu, “Understanding neural networks via rule extraction,” In IJCAI, vol. 1, pp. 480-485, 1995.
[19] S.M. Kamruzzaman and A. R. Hasan, “Rule extraction using artificial neural networks, arXiv, preprint arXiv:1009.4984, 20102010.
[20] M. Darrah, B. J. Taylor, M. Webb, and R. Livingston. “A geometric rule extraction approach used for verification and validation of a safety critical application,” in Proceedings of Florida Artificial Intelligence Research Symposium Conference, 2005.
@article{"International Journal of Information, Control and Computer Sciences:79779", author = "Osama Elsarrar and Marjorie Darrah and Richard Devin", title = "Rule Insertion Technique for Dynamic Cell Structure Neural Network ", abstract = "This paper discusses the idea of capturing an expert’s knowledge in the form of human understandable rules and then inserting these rules into a dynamic cell structure (DCS) neural network. The DCS is a form of self-organizing map that can be used for many purposes, including classification and prediction. This particular neural network is considered to be a topology preserving network that starts with no pre-structure, but assumes a structure once trained. The DCS has been used in mission and safety-critical applications, including adaptive flight control and health-monitoring in aerial vehicles. The approach is to insert expert knowledge into the DCS before training. Rules are translated into a pre-structure and then training data are presented. This idea has been demonstrated using the well-known Iris data set and it has been shown that inserting the pre-structure results in better accuracy with the same training.
", keywords = "Neural network, rule extraction, rule insertion, self-organizing map. ", volume = "14", number = "8", pages = "287-6", }
