A Hybrid Metaheuristic Framework for Evolving the PROAFTN Classifier
In this paper, a new learning algorithm based on a
hybrid metaheuristic integrating Differential Evolution (DE) and
Reduced Variable Neighborhood Search (RVNS) is introduced to train
the classification method PROAFTN. To apply PROAFTN, values of
several parameters need to be determined prior to classification. These
parameters include boundaries of intervals and relative weights for
each attribute. Based on these requirements, the hybrid approach,
named DEPRO-RVNS, is presented in this study. In some cases, the
major problem when applying DE to some classification problems
was the premature convergence of some individuals to local optima.
To eliminate this shortcoming and to improve the exploration and
exploitation capabilities of DE, such individuals were set to iteratively
re-explored using RVNS. Based on the generated results on
both training and testing data, it is shown that the performance of
PROAFTN is significantly improved. Furthermore, the experimental
study shows that DEPRO-RVNS outperforms well-known machine
learning classifiers in a variety of problems.
[1] D. Dutton and G. Conroy, "A review of machine learning," The Knowledge
Engineering Review, vol. 12:4, pp. 341-367, 1996.
[2] D. Larose, Discovering Knowledge in Data: An Introduction to Data
Mining. John Wiley & Sons, 2005.
[3] E. Alpaydin, "Introduction to machine learning (adaptive computation
and machine learning)," MIT Press, 2004.
TABLE IV
DEPRO-RVNS
TABLE V
TABLE VI
[4] B. Roy, "Multicriteria methodology for decision aiding," Kluwer Academic,
1996.
[5] N. E. Fenton and W. Wang, "Risk and confidence analysis for fuzzy
multicriteria decision making," Knowledge-Based Systems, vol. 19, no. 6,
pp. 430-437, 2006.
[6] C. Zopounidis and M. Doumpos, "Multicriteria classification and sorting
methods: A literature review," European Journal of Operational Research,
vol. 138, no. 2, pp. 229-246, 2002.
[7] K. Jabeur and A. Guitouni, "A generalized framework for
concordance/discordance-based multi-criteria classification methods,"
in Information Fusion, 2007 10th International Conference on, July
2007, pp. 1-8.
[8] N. Belacel, "Multicriteria assignment method PROAFTN: methodology
and medical application," European Journal of Operational Research,
vol. 125, no. 1, pp. 175-183, 2000.
[9] N. Belacel and M. Boulassel, "Multicriteria fuzzy assignment method:
A useful tool to assist medical diagnosis," Artificial Intelligence in
Medicine, vol. 21, no. 1-3, pp. 201-207, 2001.
[10] N. Belacel, P. Vincke, M. Scheiff, and M. Boulassel, "Acute leukemia
diagnosis aid using multicriteria fuzzy assignment methodology," Computer
Methods and Programs in Biomedicine, vol. 64, no. 2, pp. 145-
151, 2001.
[11] N. Belacel, Q. Wang, and R. Richard, "Web-integration of PROAFTN
methodology for acute leukemia diagnosis," Telemedicine Journal and
e-Health, vol. 11, no. 6, pp. 652-659, 2005.
[12] F. Al-Obeidat, N. Belacel, P. Mahanti, and J. A. Carretero, "Discretization
techniques and genetic algorithm for learning the classification
method proaftn," in Eighth International Conference On Machine Learning
and Applications. Los Alamitos, CA, USA: IEEE Computer Society,
2009, pp. 685-688.
[13] R. Storn and K. Price, "Differential evolution - A Simple and Efficient
Heuristic for Global Optimization over Continuous Spaces," Journal of
Global Optimization, vol. 11, no. 4, pp. 341-359, December 1997.
[14] S. Paterlini and T. Krink, "Differential evolution and particle swarm
optimisation in partitional clustering," Comput. Stat. Data Anal, vol. 50,
pp. 1220-1247, 2006.
[15] B. Subudhi and D. Jena, "Differential evolution and levenberg marquardt
trained neural network scheme for nonlinear system identification,"
Neural Process. Lett., vol. 27, no. 3, pp. 285-296, 2008.
[16] M. Tayel and A. H. Yassin, "An introduced neural network-differential
evolution model for small signal modeling of phemts," International
Conference on Electronic Computer Technology, vol. 0, pp. 499-506,
2009.
[17] P. Hansen and N. Mladenovic, "Variable neighborhood search for the
p-median," Location Science, vol. 5, pp. 207-226, 1997.
[18] P. Hansen and N. Mladenovic, "Variable neighborhood search: Principles
and applications," European Journal of Operational Research, no. 130,
pp. 449-467, 2001.
[19] Proceedings of the International Joint Conference on Neural Networks,
IJCNN 2008, part of the IEEE World Congress on Computational
Intelligence, WCCI 2008, Hong Kong, China, June 1-6, 2008. IEEE,
2008.
[20] C.-Y. Tsai and C.-C. Chiu, "A vns-based hierarchical clustering method,"
in CIMMACS-06: Proceedings of the 5th WSEAS International Conference
on Computational Intelligence, Man-Machine Systems and Cybernetics.
Stevens Point, Wisconsin, USA: World Scientific and
Engineering Academy and Society (WSEAS), 2006, pp. 268-275.
[21] N. Belacel, H. Raval, and A. Punnen, "Learning multicriteria fuzzy
classification method PROAFTN from data," Computers and Operations
Research, vol. 34, no. 7, pp. 1885-1898, 2007.
[22] J. R. Quinlan, "Improved use of continuous attributes in c4.5," Journal
of Artificial Intelligence Research, vol. 4, pp. 77-90, 1996.
[23] G. Cooper and E. Herskovits, "A bayesian method for the induction of
probabilistic networks from data," Machine Learning, vol. 9, no. 4, pp.
309-347, 1992.
[24] C. Burges, "A tutorial on support vector machines for pattern recognition,"
Data Mining and Knowledge Discovery, vol. 2, no. 2, pp. 1-47,
1998.
[25] G. Castellano, A. Fanelli, and M. Pelillo, "An iterative pruning algorithm
for feedforward neural networks," IEEE Transactions on Neural
Networks, vol. 8, no. 3, pp. 519-531, 1997.
[26] B. Twala, "Multiple classifier application to credit risk assessment,"
Expert Systems with Applications, vol. In Press, Uncorrected Proof, pp. -
, 2009.
[27] H. Witten, Data Mining: Practical Machine Learning Tools and Techniques.
Morgan Kaufmann Series in Data Management Systems, 2005.
[28] N. Belacel, "Multicriteria classification methods: Methodology and
medical applications," Ph.D. dissertation, Free University of Brussels,
Belgium, 1999.
[29] J. Kacprzyk, Advances in Differential Evolution. Springer, 2008.
[30] A. Asuncion and D. Newman, "UCI machine learning repository," 2007.
[31] S. Pang, D. Kim, and S. Bang, "Face membership authentication
using SVM classification tree generated by membership-based lle data
partition," IEEE Transactions on Neural Networks, vol. 16, no. 2, pp.
436-446, 2005.
[32] Y. Shirvany, M. Hayati, and R. Moradian, "Multilayer perceptron
neural networks with novel unsupervised training method for numerical
solution of the partial differential equations," Appl. Soft Comput., vol. 9,
no. 1, pp. 20-29, 2009.
[33] D. Aha, "Lazy learning," Dordrecht: Kluwer Academic Publishers, 1997.
[34] D. K. Subramanian, V. S. Ananthanarayana, and M. Narasimha Murty,
"Knowledge-based association rule mining using and-or taxonomies,"
Knowledge-Based Systems, vol. 16, no. 1, pp. 37-45, 2003.
[35] J. Demˇsar, "Statistical comparisons of classifiers over multiple data sets,"
Journal of Machine Learning Research., vol. 7, pp. 1-30, 2006.
[36] S. Garcia and F. Herrera, "An extension on "statistical comparisons of
classifiers over multiple data sets" for all pairwise comparisons," Journal
of Machine Learning Research, vol. 9, pp. 2677-2694, 2009
[1] D. Dutton and G. Conroy, "A review of machine learning," The Knowledge
Engineering Review, vol. 12:4, pp. 341-367, 1996.
[2] D. Larose, Discovering Knowledge in Data: An Introduction to Data
Mining. John Wiley & Sons, 2005.
[3] E. Alpaydin, "Introduction to machine learning (adaptive computation
and machine learning)," MIT Press, 2004.
TABLE IV
DEPRO-RVNS
TABLE V
TABLE VI
[4] B. Roy, "Multicriteria methodology for decision aiding," Kluwer Academic,
1996.
[5] N. E. Fenton and W. Wang, "Risk and confidence analysis for fuzzy
multicriteria decision making," Knowledge-Based Systems, vol. 19, no. 6,
pp. 430-437, 2006.
[6] C. Zopounidis and M. Doumpos, "Multicriteria classification and sorting
methods: A literature review," European Journal of Operational Research,
vol. 138, no. 2, pp. 229-246, 2002.
[7] K. Jabeur and A. Guitouni, "A generalized framework for
concordance/discordance-based multi-criteria classification methods,"
in Information Fusion, 2007 10th International Conference on, July
2007, pp. 1-8.
[8] N. Belacel, "Multicriteria assignment method PROAFTN: methodology
and medical application," European Journal of Operational Research,
vol. 125, no. 1, pp. 175-183, 2000.
[9] N. Belacel and M. Boulassel, "Multicriteria fuzzy assignment method:
A useful tool to assist medical diagnosis," Artificial Intelligence in
Medicine, vol. 21, no. 1-3, pp. 201-207, 2001.
[10] N. Belacel, P. Vincke, M. Scheiff, and M. Boulassel, "Acute leukemia
diagnosis aid using multicriteria fuzzy assignment methodology," Computer
Methods and Programs in Biomedicine, vol. 64, no. 2, pp. 145-
151, 2001.
[11] N. Belacel, Q. Wang, and R. Richard, "Web-integration of PROAFTN
methodology for acute leukemia diagnosis," Telemedicine Journal and
e-Health, vol. 11, no. 6, pp. 652-659, 2005.
[12] F. Al-Obeidat, N. Belacel, P. Mahanti, and J. A. Carretero, "Discretization
techniques and genetic algorithm for learning the classification
method proaftn," in Eighth International Conference On Machine Learning
and Applications. Los Alamitos, CA, USA: IEEE Computer Society,
2009, pp. 685-688.
[13] R. Storn and K. Price, "Differential evolution - A Simple and Efficient
Heuristic for Global Optimization over Continuous Spaces," Journal of
Global Optimization, vol. 11, no. 4, pp. 341-359, December 1997.
[14] S. Paterlini and T. Krink, "Differential evolution and particle swarm
optimisation in partitional clustering," Comput. Stat. Data Anal, vol. 50,
pp. 1220-1247, 2006.
[15] B. Subudhi and D. Jena, "Differential evolution and levenberg marquardt
trained neural network scheme for nonlinear system identification,"
Neural Process. Lett., vol. 27, no. 3, pp. 285-296, 2008.
[16] M. Tayel and A. H. Yassin, "An introduced neural network-differential
evolution model for small signal modeling of phemts," International
Conference on Electronic Computer Technology, vol. 0, pp. 499-506,
2009.
[17] P. Hansen and N. Mladenovic, "Variable neighborhood search for the
p-median," Location Science, vol. 5, pp. 207-226, 1997.
[18] P. Hansen and N. Mladenovic, "Variable neighborhood search: Principles
and applications," European Journal of Operational Research, no. 130,
pp. 449-467, 2001.
[19] Proceedings of the International Joint Conference on Neural Networks,
IJCNN 2008, part of the IEEE World Congress on Computational
Intelligence, WCCI 2008, Hong Kong, China, June 1-6, 2008. IEEE,
2008.
[20] C.-Y. Tsai and C.-C. Chiu, "A vns-based hierarchical clustering method,"
in CIMMACS-06: Proceedings of the 5th WSEAS International Conference
on Computational Intelligence, Man-Machine Systems and Cybernetics.
Stevens Point, Wisconsin, USA: World Scientific and
Engineering Academy and Society (WSEAS), 2006, pp. 268-275.
[21] N. Belacel, H. Raval, and A. Punnen, "Learning multicriteria fuzzy
classification method PROAFTN from data," Computers and Operations
Research, vol. 34, no. 7, pp. 1885-1898, 2007.
[22] J. R. Quinlan, "Improved use of continuous attributes in c4.5," Journal
of Artificial Intelligence Research, vol. 4, pp. 77-90, 1996.
[23] G. Cooper and E. Herskovits, "A bayesian method for the induction of
probabilistic networks from data," Machine Learning, vol. 9, no. 4, pp.
309-347, 1992.
[24] C. Burges, "A tutorial on support vector machines for pattern recognition,"
Data Mining and Knowledge Discovery, vol. 2, no. 2, pp. 1-47,
1998.
[25] G. Castellano, A. Fanelli, and M. Pelillo, "An iterative pruning algorithm
for feedforward neural networks," IEEE Transactions on Neural
Networks, vol. 8, no. 3, pp. 519-531, 1997.
[26] B. Twala, "Multiple classifier application to credit risk assessment,"
Expert Systems with Applications, vol. In Press, Uncorrected Proof, pp. -
, 2009.
[27] H. Witten, Data Mining: Practical Machine Learning Tools and Techniques.
Morgan Kaufmann Series in Data Management Systems, 2005.
[28] N. Belacel, "Multicriteria classification methods: Methodology and
medical applications," Ph.D. dissertation, Free University of Brussels,
Belgium, 1999.
[29] J. Kacprzyk, Advances in Differential Evolution. Springer, 2008.
[30] A. Asuncion and D. Newman, "UCI machine learning repository," 2007.
[31] S. Pang, D. Kim, and S. Bang, "Face membership authentication
using SVM classification tree generated by membership-based lle data
partition," IEEE Transactions on Neural Networks, vol. 16, no. 2, pp.
436-446, 2005.
[32] Y. Shirvany, M. Hayati, and R. Moradian, "Multilayer perceptron
neural networks with novel unsupervised training method for numerical
solution of the partial differential equations," Appl. Soft Comput., vol. 9,
no. 1, pp. 20-29, 2009.
[33] D. Aha, "Lazy learning," Dordrecht: Kluwer Academic Publishers, 1997.
[34] D. K. Subramanian, V. S. Ananthanarayana, and M. Narasimha Murty,
"Knowledge-based association rule mining using and-or taxonomies,"
Knowledge-Based Systems, vol. 16, no. 1, pp. 37-45, 2003.
[35] J. Demˇsar, "Statistical comparisons of classifiers over multiple data sets,"
Journal of Machine Learning Research., vol. 7, pp. 1-30, 2006.
[36] S. Garcia and F. Herrera, "An extension on "statistical comparisons of
classifiers over multiple data sets" for all pairwise comparisons," Journal
of Machine Learning Research, vol. 9, pp. 2677-2694, 2009
@article{"International Journal of Information, Control and Computer Sciences:63999", author = "Feras Al-Obeidat and Nabil Belacel and Juan A. Carretero and Prabhat Mahanti and ", title = "A Hybrid Metaheuristic Framework for Evolving the PROAFTN Classifier", abstract = "In this paper, a new learning algorithm based on a
hybrid metaheuristic integrating Differential Evolution (DE) and
Reduced Variable Neighborhood Search (RVNS) is introduced to train
the classification method PROAFTN. To apply PROAFTN, values of
several parameters need to be determined prior to classification. These
parameters include boundaries of intervals and relative weights for
each attribute. Based on these requirements, the hybrid approach,
named DEPRO-RVNS, is presented in this study. In some cases, the
major problem when applying DE to some classification problems
was the premature convergence of some individuals to local optima.
To eliminate this shortcoming and to improve the exploration and
exploitation capabilities of DE, such individuals were set to iteratively
re-explored using RVNS. Based on the generated results on
both training and testing data, it is shown that the performance of
PROAFTN is significantly improved. Furthermore, the experimental
study shows that DEPRO-RVNS outperforms well-known machine
learning classifiers in a variety of problems.", keywords = "Knowledge Discovery, Differential Evolution, Reduced Variable Neighborhood Search, Multiple criteria classification, PROAFTN, Supervised Learning.", volume = "4", number = "4", pages = "823-9", }
