Combining Diverse Neural Classifiers for Complex Problem Solving: An ECOC Approach

Combining classifiers is a useful method for solving complex problems in machine learning. The ECOC (Error Correcting Output Codes) method has been widely used for designing combining classifiers with an emphasis on the diversity of classifiers. In this paper, in contrast to the standard ECOC approach in which individual classifiers are chosen homogeneously, classifiers are selected according to the complexity of the corresponding binary problem. We use SATIMAGE database (containing 6 classes) for our experiments. The recognition error rate in our proposed method is %10.37 which indicates a considerable improvement in comparison with the conventional ECOC and stack generalization methods.

Authors:

• R. Ebrahimpour
• M. Abbasnezhad Arabi
• H. Babamiri Moghaddam

Keywords:

• Error correcting output code
• combining classifiers
• neural networks.

References:

[1] S. Haykin, Neural Networks: A Comprehensive Foundation, Prentice-
Hall, Englewood Cliffs, NJ, 1999.
[2] R.Ghaderi, Arranging simple neural network to solve complex
classification problem, PhD thesis, Surrey University, 2000.
[3] Dietterich, T.G., G. Bakiri, "Error-correcting output codes: A general
method for improving multiclass inductive learning programs", Proc. of
the 9th National Conference on Artificial Intelligence, AAAI-91. AAAI
Press, pp. 572-577, 1991.
[4] T. Windeatt, R. Ghaderi, "Binary labeling and decision level fusion",
Inform. Fusion 2, 103-112, 2001.
[5] T. Windeatt, R. Ghaderi, "Coding and decoding strategies for multi-class
learning problems", Inform. Fusion 4, 11-21, 2003.
[6] J. Kittler, S. A. Hojjatoleslami, T. Windeatt." Weighting Factors in
Multiple Expert Fusion", In Proc. Of British Machine vision Conference
BMVC 97, pp. 42-50, Essex University, Essex U.K, 1997.
[7] T.Windeatt , R. Ghaderi, "Dynamic Weighting Factors for Decision
Combining", Proc. of IEE Int. Conf. On Data Fusion, Great Malvern,
UK, pp. 123-130, October 1998.
[8] N. Hatami, R. Ebrahimpour, " Combining Multiple Classifiers: Diversify
with Boosting and Combining by Stacking" International Journal of
Computer Science and Network Security (IJCSNS), Vol.7, No.1, pp.
127-131, 2007.
[9] N. Hatami, R. Ebrahimpour, R. Ghaderi, " Error Correcting Output
Codes with Weighted Minimum Distance Decoding by Genetic
Algorithm" Proc. of the 3th Information and Knowledge Technology
(IKT07), Ferdowsi University of Mashad, Mashad, Iran, Nov. 27-29,
2007.
[10] J.Kitter , R.Ghaderi ,T.Windeatt , J.mtas "Face verification via Error
correcting output codes " International Journal of Image and Vision
Computing Vol.21,pp.1163-1169,2003.
[11] T. Windeatt and R. Ghaderi. Binary codes for multi-class decision
combining. Volume 4051, pages 23-34, Florida,USA, April 2000. 14th
Annual International Conferenceof Society of Photo-Optical
Instrumentation Engineers (SPIE).
[12] R. Ghaderi, T. Windeatt, Least squares and estimation measures via
error correcting output code, in: 2nd International Workshop on
Multiple Classifier Systems, Lecture Notes in Computer Science,
Springer-Verlag, July 2001, pp. 148-157.
[13] T.G. Dietterich, Ensemble methods in machine learning, in: J. Kittler,
F. Roli (Eds.), Multiple Classifier Systems, MCS2000,Cagliari, Italy,
Springer Lecture Notes in Computer Science, 2000,pp. 1-15.
[14] T.G Dietterich, G. Bakiri, Error-correcting output codes: A general
method for improving multiclass inductive learning programs, in:
Proceedings of the Ninth National Conference on Artificial Intelligence
(AAAI-91), AAAI Press, 1991, pp. 572-577.
[15] T.G. Dietterich, G. Bakiri, Solving multi-class learning problems via
error-correcting output codes, Journal of Artificial Intelligence Research
2 (1995) 263-286.
[16] E.B. Kong, T.G. Diettrich, Error-correcting output coding corrects bias
and variance, in: M. Kaufmann (Ed.), 12th Interna-20 T. Windeatt, R.
Ghaderi / Information Fusion 4 (2003) 11-21tional Conference of
Machine Learning, San Fransisco, 1995, pp.313-321.