Balancing Neural Trees to Improve Classification Performance
In this paper, a neural tree (NT) classifier having a
simple perceptron at each node is considered. A new concept for
making a balanced tree is applied in the learning algorithm of the
tree. At each node, if the perceptron classification is not accurate and
unbalanced, then it is replaced by a new perceptron. This separates
the training set in such a way that almost the equal number of patterns
fall into each of the classes. Moreover, each perceptron is trained only
for the classes which are present at respective node and ignore other
classes. Splitting nodes are employed into the neural tree architecture
to divide the training set when the current perceptron node repeats
the same classification of the parent node. A new error function based
on the depth of the tree is introduced to reduce the computational
time for the training of a perceptron. Experiments are performed to
check the efficiency and encouraging results are obtained in terms of
accuracy and computational costs.
[1] L. Atlas, R. Cole, Y. Muthusamy, A. Lippman, J. Connor, D. Park,
M. El-Sharkawi, and R.J. Marks, A performance comparison of trained
multilayer perceptrons and trained classification trees. Proceedings of
the IEEE, vol. 78(10), pp. 1614-1619, 1990.
[2] G. Deffuant, Neural units recruitment algorithm for generation of
decision trees. Proceedings of the International Joint Conference on
Neural Networks, San Diego, CA, vol. 1, pp. 637-642, 1990.
[3] G.L. Foresti and T. Dolso, An Adaptive High-Order Neural Tree for
Pattern Recognition, IEEE Trans. on Systemas, Man, Cybernatics- part
B: Cybernatics, Vol. 34 (2),pp. 988-996, 2004.
[4] G.L. Foresti and C. Micheloni, Generalized Neural Trees for Pattern
Classification, IEEE Trans. on Neural Networks, Vol. 13(6), pp. 1540-
1547, 2002.
[5] G. L. Foresti and G. G. Pieroni, Exploiting neural trees in range image
understanding, Pattern Recognit. Lett., vol. 19 (9), pp. 869-878, 1998.
[6] M. Kubat, Decision trees can initialize radial-basis function networks.
IEEE Transactions on Neural Networks, vol. 9(5), pp. 813-821, 1998.
[7] T. Li, L. Fang, and A. Jennings,. Structurally adaptive self-organizing
neural trees. Proceedings of the International Joint Conference on Neural
Networks, Baltimore, MD, vol. 3, pp. 329-334, 1992.
[8] R. Lippmann, An introduction to computing with neural nets, IEEE
Acoust. Speech Signal Process. Mag.,vol. 4 (2) pp. 4-22, 1987.
[9] P. Maji, Efficient Design of Neural Network Tree using a Single
Spilitting Criterion, Nerocomputing, vol. 71, pp. 787-800, 2008.
[10] P. Maji, C. Shaw, N. Ganguly, B.K. Sikdar, P.P. Chaudhuri, Theory and
application of cellular automata for pattern classification, Fundamenta
Informaticae, vol. 58(34), pp. 321354, 2003.
[11] D. Michie, D.J. Spiegelhalter and C.C. Taylor, Machine Learning, Neural
and Statistical Classification, Ellis Horwood, Chichesten, UK, 1994.
[12] S.K. Murthy,S. Kasif, and S. Salzberg, A system for induction of oblique
decision trees. Journal of Artificial Intelligence Research,vol. 2, pp. 1-
32, 1994.
[13] J.R. Quinlan, C4.5: Programs for Machine Learning, Morgan Kaufmann,
Los Atlos, CA, 1993.
[14] T.D. Sanger, A tree-structured adaptive network for function approximation
in high-dimensional spaces, IEEE Transactions on Neural Networks,
vol. 2(2), pp. 285-293, 1991.
[15] A. Sankar, and R.J. Mammone, Optimal pruning of neural tree networks
for improved generalization. Proceedings of the International Joint
Conference on Neural Networks, Seattle, WA, vol 2, pp 219- 224, 1991.
[16] I.K. Sethi, and J.H. Yoo, Structure-driven induction of decision tree
classifiers through neural learning. Pattern Recognition, vol. 30(11), pp.
1893-1904, 1997.
[17] P.E. Utgoff, N.C. Berkman, and J.A. Clouse, Decision tree induction
based on efficient tree restructuring. Machine Learning, vol. 29(1), pp.
5-44, 1997.
[18] P.E. Utgoff, and C.E. Brodley, An incremental method for finding
multivariate splits for decision trees. Proceedings of the 7th International
Conference on Machine Learning, Austin, TX: Morgan Kaufmann, pp.
58-65, 1990.
[19] Z. Zhaou and Z. Chen, Hybrid decision tree, Knowladge based systems,
vol. 15(8), pp. 515-528, 2002.
[1] L. Atlas, R. Cole, Y. Muthusamy, A. Lippman, J. Connor, D. Park,
M. El-Sharkawi, and R.J. Marks, A performance comparison of trained
multilayer perceptrons and trained classification trees. Proceedings of
the IEEE, vol. 78(10), pp. 1614-1619, 1990.
[2] G. Deffuant, Neural units recruitment algorithm for generation of
decision trees. Proceedings of the International Joint Conference on
Neural Networks, San Diego, CA, vol. 1, pp. 637-642, 1990.
[3] G.L. Foresti and T. Dolso, An Adaptive High-Order Neural Tree for
Pattern Recognition, IEEE Trans. on Systemas, Man, Cybernatics- part
B: Cybernatics, Vol. 34 (2),pp. 988-996, 2004.
[4] G.L. Foresti and C. Micheloni, Generalized Neural Trees for Pattern
Classification, IEEE Trans. on Neural Networks, Vol. 13(6), pp. 1540-
1547, 2002.
[5] G. L. Foresti and G. G. Pieroni, Exploiting neural trees in range image
understanding, Pattern Recognit. Lett., vol. 19 (9), pp. 869-878, 1998.
[6] M. Kubat, Decision trees can initialize radial-basis function networks.
IEEE Transactions on Neural Networks, vol. 9(5), pp. 813-821, 1998.
[7] T. Li, L. Fang, and A. Jennings,. Structurally adaptive self-organizing
neural trees. Proceedings of the International Joint Conference on Neural
Networks, Baltimore, MD, vol. 3, pp. 329-334, 1992.
[8] R. Lippmann, An introduction to computing with neural nets, IEEE
Acoust. Speech Signal Process. Mag.,vol. 4 (2) pp. 4-22, 1987.
[9] P. Maji, Efficient Design of Neural Network Tree using a Single
Spilitting Criterion, Nerocomputing, vol. 71, pp. 787-800, 2008.
[10] P. Maji, C. Shaw, N. Ganguly, B.K. Sikdar, P.P. Chaudhuri, Theory and
application of cellular automata for pattern classification, Fundamenta
Informaticae, vol. 58(34), pp. 321354, 2003.
[11] D. Michie, D.J. Spiegelhalter and C.C. Taylor, Machine Learning, Neural
and Statistical Classification, Ellis Horwood, Chichesten, UK, 1994.
[12] S.K. Murthy,S. Kasif, and S. Salzberg, A system for induction of oblique
decision trees. Journal of Artificial Intelligence Research,vol. 2, pp. 1-
32, 1994.
[13] J.R. Quinlan, C4.5: Programs for Machine Learning, Morgan Kaufmann,
Los Atlos, CA, 1993.
[14] T.D. Sanger, A tree-structured adaptive network for function approximation
in high-dimensional spaces, IEEE Transactions on Neural Networks,
vol. 2(2), pp. 285-293, 1991.
[15] A. Sankar, and R.J. Mammone, Optimal pruning of neural tree networks
for improved generalization. Proceedings of the International Joint
Conference on Neural Networks, Seattle, WA, vol 2, pp 219- 224, 1991.
[16] I.K. Sethi, and J.H. Yoo, Structure-driven induction of decision tree
classifiers through neural learning. Pattern Recognition, vol. 30(11), pp.
1893-1904, 1997.
[17] P.E. Utgoff, N.C. Berkman, and J.A. Clouse, Decision tree induction
based on efficient tree restructuring. Machine Learning, vol. 29(1), pp.
5-44, 1997.
[18] P.E. Utgoff, and C.E. Brodley, An incremental method for finding
multivariate splits for decision trees. Proceedings of the 7th International
Conference on Machine Learning, Austin, TX: Morgan Kaufmann, pp.
58-65, 1990.
[19] Z. Zhaou and Z. Chen, Hybrid decision tree, Knowladge based systems,
vol. 15(8), pp. 515-528, 2002.
@article{"International Journal of Information, Control and Computer Sciences:57666", author = "Asha Rani and Christian Micheloni and Gian Luca Foresti", title = "Balancing Neural Trees to Improve Classification Performance", abstract = "In this paper, a neural tree (NT) classifier having a
simple perceptron at each node is considered. A new concept for
making a balanced tree is applied in the learning algorithm of the
tree. At each node, if the perceptron classification is not accurate and
unbalanced, then it is replaced by a new perceptron. This separates
the training set in such a way that almost the equal number of patterns
fall into each of the classes. Moreover, each perceptron is trained only
for the classes which are present at respective node and ignore other
classes. Splitting nodes are employed into the neural tree architecture
to divide the training set when the current perceptron node repeats
the same classification of the parent node. A new error function based
on the depth of the tree is introduced to reduce the computational
time for the training of a perceptron. Experiments are performed to
check the efficiency and encouraging results are obtained in terms of
accuracy and computational costs.", keywords = "Neural Tree, Pattern Classification, Perceptron, Splitting
Nodes.", volume = "3", number = "7", pages = "1785-5", }
