Feature Weighting and Selection - A Novel Genetic Evolutionary Approach

A feature weighting and selection method is proposed which uses the structure of a weightless neuron and exploits the principles that govern the operation of Genetic Algorithms and Evolution. Features are coded onto chromosomes in a novel way which allows weighting information regarding the features to be directly inferred from the gene values. The proposed method is significant in that it addresses several problems concerned with algorithms for feature selection and weighting as well as providing significant advantages such as speed, simplicity and suitability for real-time systems.

Authors:

• Serkawt Khola

Keywords:

• Feature weighting
• genetic algorithm
• pattern recognition
• weightless neuron.

References:

[1] J. Austin, "RAM-Based Neural Networks, A Short History". In: James
Austin (Ed), RAM-Based Neural Networks, Progress in Neural
Computing, Vol. 9, World Scientific, London, pp 3-17, 1998.
[2] J. Bala and K. A. De Jong." Using learning to facilitate the evolution of
features for recognizing visual concepts". Evolutionary Computation
4(3): 297, 15p, 1996.
[3] B. Bhanu and Y. Lin." Genetic algorithm based feature selection for
target detection in SAR images", Image and Vision Computing, Vol. 21,
No. 7, pp. 591-608, 2003.
[4] E. R. Jr. Cacciamani." Feature Extraction and Selection using n-tuple
logics for recognition of hand-printed alphanumeric characters". PhD
thesis, Faculty of the Graduate School of Engineering and Architecture,
The Catholic University of America, 1972.
[5] S. Cost and S. Salzberg." A weighted nearest neighbour algorithm for
learning with symbolic features". Machine Learning 10(1): 57-78, 1993.
[6] D. Goldberg. "Genetic Algorithms in Search, Optimization and Machine
Learning", Addison-Wesley Longman Publishing Co., Inc., 1989
[7] E. Kalapanidas and N. Avouris. "Feature Selection for Air Quality
Forecasting: a Genetic Algorithm Approach", AI Communications
Journal, 16, (4), pp. 235 - 251, 2003.
[8] J. Kelly and L. Davis. "Hybridizing the Genetic Algorithm and the K
Nearest Neighbors Classification AIgorithm". Proceedings of the d th
International Conference on Genetic Algorithms and their Applications,
1991.
[9] L. I. Kuncheva and L. C. Jain." Designing classifier fusion systems by
genetic algorithms". IEEE Transactions On Evolutionary Computation
4(4): 327-336, 2000.
[10] A. K. Jain and D. Zongker. "Feature Selection: Evaluation, application
and small sample performance", IEEE Trans. Pattern Anal. Machine
Intell., Vol 19, pp.153-158, 1997.
[11] Z. Michaelwicz. "Genetic algorithms + data structures = evolution
programs". (2nd, extended ed.), Springer-Verlag New York, Inc., 1994
[12] W. F. Punch, E. D. Goodman, M. Pei, L. Chia-Shun, P. Hovland, R.
Enbody. "Further research on feature selection and classi cation using
genetic algorithms". In Proceedings of the Fifth International Conference
on Genetic Algorithms, pages 557-564, Palo Alto, CA, USA, 1993.
[13] M. L. Raymer, W. F. Punch, E. D. Goodman, L. A. Kuhn, A. K. Jain.
"Dimensionality reduction using genetic algorithms". IEEE Transactions
On Evolutionary Computation 4(2): 164-171, 2000.
[14] W. Siedlecki and J. Sklansky. "A Note On Genetic Algorithms For
Large-Scale Feature-Selection". Pattern Recognition Letters 10(5): 335-
347, 1989.
[15] Z. H. Sun, G. Bebis, R. Miller. "Object detection using feature subset
selection", Pattern Recognition 37(11): 2165-2176, 2004.
[16] F. E. H. Tay and L. J. Cao. "A comparative study of saliency analysis
and genetic algorithm for feature selection in support vector machines".
Intelligent Data Analysis 5(3): 191, 19p, 2001.
[17] S. Theodoridis and K. Koutroumbas."Pattern Recognition". Second
Edition, 2003.
[18] J. Yang and V. G. Honavar. "Feature Subset Selection Using a Genetic
Algorithm". IEEE Intelligent Systems 13(2): 44-49, 1998.
[19] S. Khola, "System and Method for Information Retrieval", Patent
GB1100947.9, Patent Pending, 2011.
[20] S. Khola, "Genetic Evolution using Weighted Sub-spaces and
Weightless Neurons for Pattern Recognition", PhD Thesis, University
Kent, Canterbury, United Kingdom, 2006.
[21] N. D. Smith, S. Khola and K. Sirlantzis. "Classifier Selection using a
Weightless Neuron-Based Genetic Algorithm". Proceedings of the 6th
International Conference on Recent Advances in Soft Computing
(RASC2006), IEE, ENNS, EUSPLAT, IAPR, Canterbury, Kent, UK.
July 10-12, 2006 .
[22] MFEAT, http://www.ics.uci.edu/~mlearn/MLSummary.html Dataset by:
Robert P.W. Duin, Department of Applied Physics, Delft University of
Technology P.O. Box 5046, 2600 GA Delft, The Netherlands.Email:
[email protected] http : //www.ph.tn.tudelft.nl/~duin [1] tel +31 15
2786143.