Integration of Support Vector Machine and Bayesian Neural Network for Data Mining and Classification

Several combinations of the preprocessing algorithms, feature selection techniques and classifiers can be applied to the data classification tasks. This study introduces a new accurate classifier, the proposed classifier consist from four components: Signal-to- Noise as a feature selection technique, support vector machine, Bayesian neural network and AdaBoost as an ensemble algorithm. To verify the effectiveness of the proposed classifier, seven well known classifiers are applied to four datasets. The experiments show that using the suggested classifier enhances the classification rates for all datasets.

Authors:

• Essam Al-Daoud

Keywords:

• AdaBoost
• Bayesian neural network
• Signal-to-Noise
• support vector machine
• MCMC.

References:

[1] T.M. Mitchell, Machine learning. McGraw-Hill, New York, NY, USA,
1997.
[2] E. Al-Daoud, "A comparison between three neural network models for
classification problems," Journal of Artificial Intelligence, vol. 2, no. 2,
pp. 56-64, 2009.
[3] E. Al-Daoud, "Identifying DNA splice sites using patterns statistical
properties and fuzzy neural networks," EXCLI Journal, vol. 8, pp. 195-
202, 2009.
[4] P. Venkatesan and S. Anitha, "Application of a radial basis function
neural network for diagnosis of diabetes mellitus," Current Science, vol.
91, no. 9, pp. 1195-1199, Nov. 2006.
[5] R. Bouckaert, "Naive Bayes classifiers that perform well with
continuous variables," Lecture Notes in Computer Science, vol 3339, pp.
1089-1094, 2004.
[6] I. Steinwart and A. Christmann, Support Vector Machines. Springer,
New York, 2008.
[7] R. Polikar, "Ensemble based systems in decision making," IEEE
Circuits and Systems Magazine, vol. 3, pp. 21-45, 2006.
[8] R. M. Neal, Bayesian learning of neural networks, Springer-Verlag,
New York, 1996.
[9] Z. Waszczyszyn and L. Ziemiański, "Neurocomputing in the analysis of
selected inverse problems of mechanics of structures and materials,"
Computer Assisted Mechanics and Engineering Sciences, vol. 13, no. 1
pp. 125-159, 2006. www.lce.hut.fi/research/mm/mcmcstuff.
[10] C. W. Hsu, C. C. Chang and C. J. Lin. "A practical guide to support
vector classification," Technical report, Department of Computer
Science, National Taiwan University. July, 2003.
[11] H. B. Hashemi, A. Shakery and M. P. Naeini, "Protein fold pattern
recognition using bayesian ensemble of RBF neural networks" 2009
International Conference of Soft Computing and Pattern Recognition,
pp. 436-441. 2009.
[12] CLOP Package http://www.modelselect.inf.ethz.ch