Judges System for Classifiers Specialization

In this paper we designed and implemented a new ensemble of classifiers based on a sequence of classifiers which were specialized in regions of the training dataset where errors of its trained homologous are concentrated. In order to separate this regions, and to determine the aptitude of each classifier to properly respond to a new case, it was used another set of classifiers built hierarchically. We explored a selection based variant to combine the base classifiers. We validated this model with different base classifiers using 37 training datasets. It was carried out a statistical comparison of these models with the well known Bagging and Boosting, obtaining significantly superior results with the hierarchical ensemble using Multilayer Perceptron as base classifier. Therefore, we demonstrated the efficacy of the proposed ensemble, as well as its applicability to general problems.

Authors:

• Abdel Rodríguez
• Isis Bonet
• Ricardo Grau
• María M. García

Keywords:

• classifiers
• delegation
• ensemble

References:

[1] L. Breiman, Bagging predictors. Machine Learning, 1996. 24: p. 123-
140.
[2] Y. Freund and R. E. Schapire, Experiments with a new boosting
algorithm. Thirteenth International Conference on Machine Learning,
1996: p. 148-156.
[3] R. E. Schapire, The strength of weak learnability. Machine Learning,
1990. 5(2): p. 197-227.
[4] R. A. Jacobs, S. J. Nowlan, and G. E. Hinton, Adaptative mixtures of
local experts. Neural Computation, 1991. 3: p. 79-87.
[5] M. J. Jordan and R. A. Jacobs, Hirarchical mixtures of experts and the
EM algorithm. Neural Computation, 1994. 6: p. 79-87.
[6] D. Wolpert, Stacked generalization. Neural Networks, 1992. 5(2): p.
241-259.
[7] D. J. N. A. Asuncion. UCI Machine Learning Repository. 2007.
[8] C. Ferri, P. Flach, and J. Hernández-Orallo. Delagating Classifiers. in
21st International Conference on Machine Learning. 2004. Canada.