Fuzzy Rules Generation and Extraction from Support Vector Machine Based on Kernel Function Firing Signals
Our study proposes an alternative method in building
Fuzzy Rule-Based System (FRB) from Support Vector Machine
(SVM). The first set of fuzzy IF-THEN rules is obtained through
an equivalence of the SVM decision network and the zero-ordered
Sugeno FRB type of the Adaptive Network Fuzzy Inference System
(ANFIS). The second set of rules is generated by combining the
first set based on strength of firing signals of support vectors using
Gaussian kernel. The final set of rules is then obtained from the
second set through input scatter partitioning. A distinctive advantage
of our method is the guarantee that the number of final fuzzy IFTHEN
rules is not more than the number of support vectors in the
trained SVM. The final FRB system obtained is capable of performing
classification with results comparable to its SVM counterpart, but it
has an advantage over the black-boxed SVM in that it may reveal
human comprehensible patterns.
[1] S. Aeberhard, D. Coomans, and O. de Vel, "The Classification Performance
of RDA," Tech. Rep., no. 92-01, 1992.
[2] S. Ali and K. A. Smith-Miles, "Improved Support Vector Machine
Generalization Using Normalized Input Space," In: A. Sattar and B.H.
Kang (Eds.), Artificial Intelligence, LNAI 4304, Springer-Verlag, Berlin,
Heidelberg, pp. 362-371, 2006.
[3] R. Andrews, J. Diederich, and A. B. Tickle, "Survey and critique of
techniques for extracting rules from trained artificial neural networks,"
Knowledge-Based Systems, vol. 8, no. 6, pp. 373-389, 1995.
[4] N. Barakat and J. Diederich, "Eclectic Rule-Extraction from Support
Vector Machines," International Journal of Computational Intelligence,
vol. 2, no. 1, pp. 59-62, 2005.
[5] C. M. Bishop, Neural Networks for Pattern Recognition. Oxford University
Press, New York, 1995, pp. 372-375.
[6] C. J. C. Burges, "A tutorial on support vector machines for pattern
recognition," Data Mining and Knowledge Discovery, vol. 2, pp. 121-
167, 1998.
[7] C. Cortes and V. N. Vapnik, "Support Vector Networks," Machine
Learning, vol. 20, pp. 273-297, 1995.
[8] J. Diederich (Ed.), "Rule Extraction from Support Vector Machines,"
Studies in Computational Intelligence, vol. 80, Springer-Verlag, Berlin,
Heidelberg, 2008, pp. 3-30.
[9] R. A. Fisher, "The use of multiple measurements in taxonomic problems,"
Annual Eugenics, vol. 7, Part II, pp. 179-188, 1936.
[10] G. Fung, S. Sandilya, and R. B. Rao, "Rule extraction from linear
support vector machines," in Proceedings of the 11th ACM SIGKDD
international Conference on Knowledge Discovery in Data Mining, 2005,
pp. 32-40.
[11] S. J. Haberman, "Generalized Residuals for Log-Linear Models," in
Proceedings of the 9th International Biometrics Conference, Boston,
1976, pp. 104-122.
[12] S. Haykin, Neural Networks, A Comprehensive Foundation, Macmillan,
New York, NY, 1994.
[13] J. Huysmans, B. Baesens, and J. Vanthienen, "ITER: an algorithm for
predictive regression rule extraction," in 8th International Conference on
Data Warehousing and Knowledge Discovery, Springer Verlag, lncs 4081,
2006, pp. 270-279.
[14] J. Huysmans, R. Setiono, B. Baesens, and J. Vanthienen, "Minerva:
Sequential Covering for Rule Extraction," IEEE Transactions on Systems,
Man, and Cybernetics Part B: Cybernetics, vol. 38, no. 2, pp. 299-309,
2008.
[15] J. -S. R. Jang and C. -T. Sun, "Functional equivalence between radial
basis function networks and fuzzy inference systems," IEEE Trans. Neural
Networks, vol. 4, pp. 156-158, 1992.
[16] J. Jang, C. Sun, and E. Mizutani, Neuro-Fuzzy and Soft Computing.
Prentice Hall International, 1997, pp. 333-342.
[17] E. Kolman and M. Margaliot, "Are artificial neural networks white
boxes?" IEEE Trans. Neural Networks, vol. 16, no. 4, pp. 844-852, 2005.
[18] S. Kumar,Neural Networks: A Classroom Approach. McGraw-Hill,
International Edition, 2005, pp. 273-304.
[19] K. R. Muller, A. J. Smola, G. Ratsch, B. Scholkopf, J. Kohlmorgen and
V. N. Vapnik, "Predicting time series with support vector machines," in
ICANN, 1997, pp. 999-1004.
[20] H. Nunez, C. Angulo, and A. Catala, "Rule Extraction Based on Support
and Prototype Vectors," in Studies in Computational Intelligence, vol. 80,
Springer-Verlag, Berlin, Heidelberg, 2008 pp. 109-134.
[21] V. J. Sigillito, S. P. Wing, L. V. Hutton, and K. B. Baker, "Classification
of radar returns from the ionosphere using neural networks," Johns
Hopkins APL Technical Digest, vol. 10, pp. 262-266, 1989.
[22] S. Sivanandam, S. Sumathi, and S. Deepa, Introduction to Fuzzy Logic
using MATLAB. Springer Verlag, Berlin, Heidelberg, 2007.
[23] V. N. Vapnik, Statistical Learning Theory. John Wiley & Sons, 1998,
pp. 375-520.
[24] W. H. Wolberg and O. L. Mangasarian, "Multisurface method of pattern
separation for medical diagnosis applied to breast cytology," Proceedings
of the National Academy of Sciences, vol. 87, pp. 9193-9196, 1990.
[1] S. Aeberhard, D. Coomans, and O. de Vel, "The Classification Performance
of RDA," Tech. Rep., no. 92-01, 1992.
[2] S. Ali and K. A. Smith-Miles, "Improved Support Vector Machine
Generalization Using Normalized Input Space," In: A. Sattar and B.H.
Kang (Eds.), Artificial Intelligence, LNAI 4304, Springer-Verlag, Berlin,
Heidelberg, pp. 362-371, 2006.
[3] R. Andrews, J. Diederich, and A. B. Tickle, "Survey and critique of
techniques for extracting rules from trained artificial neural networks,"
Knowledge-Based Systems, vol. 8, no. 6, pp. 373-389, 1995.
[4] N. Barakat and J. Diederich, "Eclectic Rule-Extraction from Support
Vector Machines," International Journal of Computational Intelligence,
vol. 2, no. 1, pp. 59-62, 2005.
[5] C. M. Bishop, Neural Networks for Pattern Recognition. Oxford University
Press, New York, 1995, pp. 372-375.
[6] C. J. C. Burges, "A tutorial on support vector machines for pattern
recognition," Data Mining and Knowledge Discovery, vol. 2, pp. 121-
167, 1998.
[7] C. Cortes and V. N. Vapnik, "Support Vector Networks," Machine
Learning, vol. 20, pp. 273-297, 1995.
[8] J. Diederich (Ed.), "Rule Extraction from Support Vector Machines,"
Studies in Computational Intelligence, vol. 80, Springer-Verlag, Berlin,
Heidelberg, 2008, pp. 3-30.
[9] R. A. Fisher, "The use of multiple measurements in taxonomic problems,"
Annual Eugenics, vol. 7, Part II, pp. 179-188, 1936.
[10] G. Fung, S. Sandilya, and R. B. Rao, "Rule extraction from linear
support vector machines," in Proceedings of the 11th ACM SIGKDD
international Conference on Knowledge Discovery in Data Mining, 2005,
pp. 32-40.
[11] S. J. Haberman, "Generalized Residuals for Log-Linear Models," in
Proceedings of the 9th International Biometrics Conference, Boston,
1976, pp. 104-122.
[12] S. Haykin, Neural Networks, A Comprehensive Foundation, Macmillan,
New York, NY, 1994.
[13] J. Huysmans, B. Baesens, and J. Vanthienen, "ITER: an algorithm for
predictive regression rule extraction," in 8th International Conference on
Data Warehousing and Knowledge Discovery, Springer Verlag, lncs 4081,
2006, pp. 270-279.
[14] J. Huysmans, R. Setiono, B. Baesens, and J. Vanthienen, "Minerva:
Sequential Covering for Rule Extraction," IEEE Transactions on Systems,
Man, and Cybernetics Part B: Cybernetics, vol. 38, no. 2, pp. 299-309,
2008.
[15] J. -S. R. Jang and C. -T. Sun, "Functional equivalence between radial
basis function networks and fuzzy inference systems," IEEE Trans. Neural
Networks, vol. 4, pp. 156-158, 1992.
[16] J. Jang, C. Sun, and E. Mizutani, Neuro-Fuzzy and Soft Computing.
Prentice Hall International, 1997, pp. 333-342.
[17] E. Kolman and M. Margaliot, "Are artificial neural networks white
boxes?" IEEE Trans. Neural Networks, vol. 16, no. 4, pp. 844-852, 2005.
[18] S. Kumar,Neural Networks: A Classroom Approach. McGraw-Hill,
International Edition, 2005, pp. 273-304.
[19] K. R. Muller, A. J. Smola, G. Ratsch, B. Scholkopf, J. Kohlmorgen and
V. N. Vapnik, "Predicting time series with support vector machines," in
ICANN, 1997, pp. 999-1004.
[20] H. Nunez, C. Angulo, and A. Catala, "Rule Extraction Based on Support
and Prototype Vectors," in Studies in Computational Intelligence, vol. 80,
Springer-Verlag, Berlin, Heidelberg, 2008 pp. 109-134.
[21] V. J. Sigillito, S. P. Wing, L. V. Hutton, and K. B. Baker, "Classification
of radar returns from the ionosphere using neural networks," Johns
Hopkins APL Technical Digest, vol. 10, pp. 262-266, 1989.
[22] S. Sivanandam, S. Sumathi, and S. Deepa, Introduction to Fuzzy Logic
using MATLAB. Springer Verlag, Berlin, Heidelberg, 2007.
[23] V. N. Vapnik, Statistical Learning Theory. John Wiley & Sons, 1998,
pp. 375-520.
[24] W. H. Wolberg and O. L. Mangasarian, "Multisurface method of pattern
separation for medical diagnosis applied to breast cytology," Proceedings
of the National Academy of Sciences, vol. 87, pp. 9193-9196, 1990.
@article{"International Journal of Information, Control and Computer Sciences:53870", author = "Prasan Pitiranggon and Nunthika Benjathepanun and Somsri Banditvilai and Veera Boonjing", title = "Fuzzy Rules Generation and Extraction from Support Vector Machine Based on Kernel Function Firing Signals", abstract = "Our study proposes an alternative method in building
Fuzzy Rule-Based System (FRB) from Support Vector Machine
(SVM). The first set of fuzzy IF-THEN rules is obtained through
an equivalence of the SVM decision network and the zero-ordered
Sugeno FRB type of the Adaptive Network Fuzzy Inference System
(ANFIS). The second set of rules is generated by combining the
first set based on strength of firing signals of support vectors using
Gaussian kernel. The final set of rules is then obtained from the
second set through input scatter partitioning. A distinctive advantage
of our method is the guarantee that the number of final fuzzy IFTHEN
rules is not more than the number of support vectors in the
trained SVM. The final FRB system obtained is capable of performing
classification with results comparable to its SVM counterpart, but it
has an advantage over the black-boxed SVM in that it may reveal
human comprehensible patterns.", keywords = "Fuzzy Rule Base, Rule Extraction, Rule Generation,Support Vector Machine.", volume = "4", number = "8", pages = "1222-8", }
