Ensemble Learning with Decision Tree for Remote Sensing Classification
In recent years, a number of works proposing the
combination of multiple classifiers to produce a single
classification have been reported in remote sensing literature. The
resulting classifier, referred to as an ensemble classifier, is
generally found to be more accurate than any of the individual
classifiers making up the ensemble. As accuracy is the primary
concern, much of the research in the field of land cover
classification is focused on improving classification accuracy. This
study compares the performance of four ensemble approaches
(boosting, bagging, DECORATE and random subspace) with a
univariate decision tree as base classifier. Two training datasets,
one without ant noise and other with 20 percent noise was used to
judge the performance of different ensemble approaches. Results
with noise free data set suggest an improvement of about 4% in
classification accuracy with all ensemble approaches in
comparison to the results provided by univariate decision tree
classifier. Highest classification accuracy of 87.43% was achieved
by boosted decision tree. A comparison of results with noisy data
set suggests that bagging, DECORATE and random subspace
approaches works well with this data whereas the performance of
boosted decision tree degrades and a classification accuracy of
79.7% is achieved which is even lower than that is achieved (i.e.
80.02%) by using unboosted decision tree classifier.
[1] G. Giacinto and F. Roli, Ensembles of neural networks for soft
classification of remote sensing images. Proceedings of the European
Symposium on Intelligent Techniques, European Network for Fuzzy
Logic and Uncertainty Modelling in Information Technology, Bari,
Italy, 166-170, 1997.
[2] J. A.,Benediktsson, J. R.,Sveinsson, O. K.,Ersoy and P. H., Swain,
Parallel consensual neural networks. IEEE Trans. Neural Networks, 8,
1997, 54-65.
[3] F. Roli, G. Giacinto and G. Vernazza, Comparison and combination
of statistical and neural networks algorithms for remote-sensing
image classification. Neurocomputation in Remote Sensing Data
Analysis, Austin, J., Kanellopoulos, I., Roli, F. and Wilkinson G.
(Eds.), Berlin: Springer-Verlag, 117-124, 1997.
[4] L., Breiman, Bagging predictors, Machine Learning, 26, 1996, 123-
140.
[5] Y.Freund and R. Schapire, Experiments with a new boosting
algorithm. Machine Learning: Proceedings of the Thirteenth
International conference, 148-156, 1996.
[6] M. A. Friedl, C. E. Brodley, and A. H. Strahler, Maximizing land
cover classification accuracies produced by decision tree at
continental to global scales. IEEE Transactions on Geoscience and
Remote Sensing. 37, 1999, 969-977.
[7] M. Pal and P. M. Mather, Decision tree classifiers and land use
classification. Proceedings of the 27th Annual Conference of the
Remote Sensing Society, 12-14 September, London, UK, 2001.
[8] G. J., Briem, J. A.,Benediktsson, and J. R., Sveinsson, Multiple
Classifiers Applied to Multisource Remote Sensing Data IEEE
Transactions on Geoscience and Remote Sensing, 40, 2002, 2291-
2299.
[9] P. Melville and R. Mooney, Constructing diverse classifier ensembles
using artificial training examples. In Proceedings of the Eighteenth
International Joint Conference on Artificial Intelligence, 505-510,
2003, Acapulco, Mexico, August.
[10] T.K.Ho, The Random Subspace Method for Constructing Decision
Forests. IEEE Transactions on Pattern Analysis and Machine
Intelligence 20, 1998, 832-844.
[11] L. Hansen and P. Salamon, Neural network ensembles, IEEE
Transactions on Pattern recognition and Machine intelligence, 12,
1990, 993-1001.
[12] A. Krogh and J. Vedelsby, Neural networks ensembles, cross
validation and active learning. In D.S. Touretzky, G. Tesauro, and
T.K. Leen, editors, Advances in Neural Information Processing
Systems, volume 7, pages 107-115, 1995, MIT Press, Cambridge,
MA.
[13] L.,Breiman, J.H. Friedman, R.A.,Olshen and C.J.,Stone,
Classification and Regression Trees, Wadsworth, Monterey, CA,
1984.
[14] S. K.Murthy, S. Kasif and S. Salzberg, A system for induction of
oblique decision trees. Journal of Artificial Intelligence Research, 2,
1994, 1-32.
[15] I. Kononenko and J. S. Hong, Attribute selection for modelling.
Future Generation Computer Systems, 13, 1997, 181-195.
[16] J. Mingers, An empirical comparison of selection measures for
decision tree induction. Machine Learning, 3, 1989, 319-342.
[17] J. R. Quinlan, C4.5: Programs for Machine Learning. San Mateo:
Morgan Kaufmann, San Francisco, 1993.
[1] G. Giacinto and F. Roli, Ensembles of neural networks for soft
classification of remote sensing images. Proceedings of the European
Symposium on Intelligent Techniques, European Network for Fuzzy
Logic and Uncertainty Modelling in Information Technology, Bari,
Italy, 166-170, 1997.
[2] J. A.,Benediktsson, J. R.,Sveinsson, O. K.,Ersoy and P. H., Swain,
Parallel consensual neural networks. IEEE Trans. Neural Networks, 8,
1997, 54-65.
[3] F. Roli, G. Giacinto and G. Vernazza, Comparison and combination
of statistical and neural networks algorithms for remote-sensing
image classification. Neurocomputation in Remote Sensing Data
Analysis, Austin, J., Kanellopoulos, I., Roli, F. and Wilkinson G.
(Eds.), Berlin: Springer-Verlag, 117-124, 1997.
[4] L., Breiman, Bagging predictors, Machine Learning, 26, 1996, 123-
140.
[5] Y.Freund and R. Schapire, Experiments with a new boosting
algorithm. Machine Learning: Proceedings of the Thirteenth
International conference, 148-156, 1996.
[6] M. A. Friedl, C. E. Brodley, and A. H. Strahler, Maximizing land
cover classification accuracies produced by decision tree at
continental to global scales. IEEE Transactions on Geoscience and
Remote Sensing. 37, 1999, 969-977.
[7] M. Pal and P. M. Mather, Decision tree classifiers and land use
classification. Proceedings of the 27th Annual Conference of the
Remote Sensing Society, 12-14 September, London, UK, 2001.
[8] G. J., Briem, J. A.,Benediktsson, and J. R., Sveinsson, Multiple
Classifiers Applied to Multisource Remote Sensing Data IEEE
Transactions on Geoscience and Remote Sensing, 40, 2002, 2291-
2299.
[9] P. Melville and R. Mooney, Constructing diverse classifier ensembles
using artificial training examples. In Proceedings of the Eighteenth
International Joint Conference on Artificial Intelligence, 505-510,
2003, Acapulco, Mexico, August.
[10] T.K.Ho, The Random Subspace Method for Constructing Decision
Forests. IEEE Transactions on Pattern Analysis and Machine
Intelligence 20, 1998, 832-844.
[11] L. Hansen and P. Salamon, Neural network ensembles, IEEE
Transactions on Pattern recognition and Machine intelligence, 12,
1990, 993-1001.
[12] A. Krogh and J. Vedelsby, Neural networks ensembles, cross
validation and active learning. In D.S. Touretzky, G. Tesauro, and
T.K. Leen, editors, Advances in Neural Information Processing
Systems, volume 7, pages 107-115, 1995, MIT Press, Cambridge,
MA.
[13] L.,Breiman, J.H. Friedman, R.A.,Olshen and C.J.,Stone,
Classification and Regression Trees, Wadsworth, Monterey, CA,
1984.
[14] S. K.Murthy, S. Kasif and S. Salzberg, A system for induction of
oblique decision trees. Journal of Artificial Intelligence Research, 2,
1994, 1-32.
[15] I. Kononenko and J. S. Hong, Attribute selection for modelling.
Future Generation Computer Systems, 13, 1997, 181-195.
[16] J. Mingers, An empirical comparison of selection measures for
decision tree induction. Machine Learning, 3, 1989, 319-342.
[17] J. R. Quinlan, C4.5: Programs for Machine Learning. San Mateo:
Morgan Kaufmann, San Francisco, 1993.
@article{"International Journal of Information, Control and Computer Sciences:49854", author = "Mahesh Pal", title = "Ensemble Learning with Decision Tree for Remote Sensing Classification", abstract = "In recent years, a number of works proposing the
combination of multiple classifiers to produce a single
classification have been reported in remote sensing literature. The
resulting classifier, referred to as an ensemble classifier, is
generally found to be more accurate than any of the individual
classifiers making up the ensemble. As accuracy is the primary
concern, much of the research in the field of land cover
classification is focused on improving classification accuracy. This
study compares the performance of four ensemble approaches
(boosting, bagging, DECORATE and random subspace) with a
univariate decision tree as base classifier. Two training datasets,
one without ant noise and other with 20 percent noise was used to
judge the performance of different ensemble approaches. Results
with noise free data set suggest an improvement of about 4% in
classification accuracy with all ensemble approaches in
comparison to the results provided by univariate decision tree
classifier. Highest classification accuracy of 87.43% was achieved
by boosted decision tree. A comparison of results with noisy data
set suggests that bagging, DECORATE and random subspace
approaches works well with this data whereas the performance of
boosted decision tree degrades and a classification accuracy of
79.7% is achieved which is even lower than that is achieved (i.e.
80.02%) by using unboosted decision tree classifier.", keywords = "Ensemble learning, decision tree, remote sensingclassification.", volume = "1", number = "12", pages = "3755-3", }