Unsupervised Clustering Methods for Identifying Rare Events in Anomaly Detection
It is important problems to increase the detection rates
and reduce false positive rates in Intrusion Detection System (IDS).
Although preventative techniques such as access control and
authentication attempt to prevent intruders, these can fail, and as a
second line of defence, intrusion detection has been introduced. Rare
events are events that occur very infrequently, detection of rare
events is a common problem in many domains. In this paper we
propose an intrusion detection method that combines Rough set and
Fuzzy Clustering. Rough set has to decrease the amount of data and
get rid of redundancy. Fuzzy c-means clustering allow objects to
belong to several clusters simultaneously, with different degrees of
membership. Our approach allows us to recognize not only known
attacks but also to detect suspicious activity that may be the result of
a new, unknown attack. The experimental results on Knowledge
Discovery and Data Mining-(KDDCup 1999) Dataset show that the
method is efficient and practical for intrusion detection systems.
[1] R. Bace and P. Mell, "Intrusion Detection Systems", NIST Special
Publications SP 800. 31 November 2001.
[2] D. Denning, "An intrusion-detection model," In IEEE computer society
symposium on research in security and privacy, 1986, pp. 118-131.
[3] T. Lane, "Machine Learning techniques for the computer security", PhD
thesis, Purdue University, 2000.
[4] W. Lee and S. Stolfo, "Data mining approaches for intrusion
detection," Proceedings of the 7th USENIX security symposium, , 1998.
[5] D. Dagupta and F. Gonzalez, "An immunity-based technique to
characterize intrusions in computer networks", IEEE Transactions on
Evolutionary Computation, Vol. 6, June 2002, pp.28- 291.
[6] H. Jin, J. Sun, H. Chen, and Z. Han, "A Fuzzy Data Mining based
Intrusion Detection System", Proceedings of 10thInternational Workshop
on future Trends in Distributed Computing Systems (FTDCS04) IEEE
Computer Society, Suzhou, China, May 26-28, 2004, pp. 191-197.
[7] J. Twycross, "Immune Systems, Danger Theory and Intrusion
Detection", presented at the AISB 2004. Symposium on Immune System
and Cognition, Leeds, U.K., March 2004.
[8] R.T. Alves, M.R.B.S. Delgado, H.S. Lopes, A.A. Freitas, "An artificial
immune system for fuzzy-rule induction in data mining", Lecture Notes
in Computer Science, Berlin: Springer Verlag, v. 3242, 2004, pp. 1011-
1020.
[9] W. Lee, S. Stolfo, and K. Mok, "A data mining framework for building
intrusion detection models", Proceedings of the 1999 IEEE Symposium
on Security and Privacy, May 1999, pp.120-132.
[10] A. Lazarevic, A. Ozgur, L. Ertoz, J. Srivastava, and V. Kumar, "A
comparative study of anomaly detection schemes in network intrusion
detection", In SIAM International Conference on Data Mining, 2003.
[11] R. Jensen and Q. Shen, "Rough and fuzzy sets for dimensionality
reduction", Proceedings of the 2001 UK Workshop on Computational
Intelligence, 2001, pp. 69-74.
[12] D. Sarjon and Mohd Noor Md Sap, "Association Rules using Rough Set
and Association Rule Methods", Proceedings of 7th Pacific Rim
International Conference on Artificial Intelligence (PRICAI-02), Tokyo,
Japan, August 18-22, 2002, pp. 238-243.
[13] S. Theodoridis, K. Koutroubas, "Pattern recognition", Academic Press,
1999.
[14] S. Albayrak, Fatih Amasyali, Fuzzy c-means clustering on Medical
Diagnostic Systems, International XII. Turkish Symposium on Artificial
Intelligence and Neural Networks, TAINN 2003.
[15] J. Bezkek, "Pattern Recognition with Fuzzy Objective Function
Algorithms", Plenum Press, USA, 1981.
[16] KDD data set, 1999;
http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html
[17] P. Laskov, K. Rieck, C. Schäfer, K.R. Müller, "Visualization of
anomaly detection using prediction sensitivity", Proceeding of
Sicherheit, April 2005, pp. 197-208.
[18] MathWorks, Statistical Toolbox for User-s Guide, MathWorks, 2001.
[19] A. Ôêà hrm, "ROSETTA Technical Reference Manual", Department of
Computer and Information Science, Norwegian University of Science
and Technology (NTNU), Trondheim, Norway, 2000.
[20] W. Chimphlee, Abdul Hanan Abdullah, Mohd Noor Md Sap and S.
Chimphlee, "Unsupervised Anomaly Detection with Unlabeled Data
using Clustering", Proc. Int. Conf. on ICT- Mercu Buana ICT2005. pp.
42-49.
[21] A. Lazarevic, A. Ozgur, L. Ertoz, J. Srivastava, and V. Kumar, "A
comparative study of anomaly detection schemes in network intrusion
detection". In SIAM; International Conference on Data Mining, 2003.
[22] Zhengxin Chen, Data Mining and Uncertain Reasoning - An Integrated
Approach, Wiley, 2001.
[23] Fernando Godínez, Dieter Hutter, Raul Monroy "Attribute Reduction for
Effective Intrusion Detection". AWIC 2004: 74-83.
[24] W. Chimphlee, Abdul Hanan Abdullah, Mohd Noor Md Sap,
S.Chimphlee, and S. Srinoy, Unsupervised Anomaly Detection without
Prior Knowledge Using Clustering, International workshop on
information Technology 2005 (IAIT2005), 25-26 November 2005.,
Thailand.
[25] W. Chimphlee, Mohd Noor Md Sap, Abdul Hanan Abdullah, and S.
Chimphlee, Semi-Supervised Learning to Identify Suspicious Activity
for Anomaly Detection, 3rd International Conference on Computational
Intelligence, Robotics and Autonomous Systems (CIRAS2005), 13-16
December 2005, Singapore.
[1] R. Bace and P. Mell, "Intrusion Detection Systems", NIST Special
Publications SP 800. 31 November 2001.
[2] D. Denning, "An intrusion-detection model," In IEEE computer society
symposium on research in security and privacy, 1986, pp. 118-131.
[3] T. Lane, "Machine Learning techniques for the computer security", PhD
thesis, Purdue University, 2000.
[4] W. Lee and S. Stolfo, "Data mining approaches for intrusion
detection," Proceedings of the 7th USENIX security symposium, , 1998.
[5] D. Dagupta and F. Gonzalez, "An immunity-based technique to
characterize intrusions in computer networks", IEEE Transactions on
Evolutionary Computation, Vol. 6, June 2002, pp.28- 291.
[6] H. Jin, J. Sun, H. Chen, and Z. Han, "A Fuzzy Data Mining based
Intrusion Detection System", Proceedings of 10thInternational Workshop
on future Trends in Distributed Computing Systems (FTDCS04) IEEE
Computer Society, Suzhou, China, May 26-28, 2004, pp. 191-197.
[7] J. Twycross, "Immune Systems, Danger Theory and Intrusion
Detection", presented at the AISB 2004. Symposium on Immune System
and Cognition, Leeds, U.K., March 2004.
[8] R.T. Alves, M.R.B.S. Delgado, H.S. Lopes, A.A. Freitas, "An artificial
immune system for fuzzy-rule induction in data mining", Lecture Notes
in Computer Science, Berlin: Springer Verlag, v. 3242, 2004, pp. 1011-
1020.
[9] W. Lee, S. Stolfo, and K. Mok, "A data mining framework for building
intrusion detection models", Proceedings of the 1999 IEEE Symposium
on Security and Privacy, May 1999, pp.120-132.
[10] A. Lazarevic, A. Ozgur, L. Ertoz, J. Srivastava, and V. Kumar, "A
comparative study of anomaly detection schemes in network intrusion
detection", In SIAM International Conference on Data Mining, 2003.
[11] R. Jensen and Q. Shen, "Rough and fuzzy sets for dimensionality
reduction", Proceedings of the 2001 UK Workshop on Computational
Intelligence, 2001, pp. 69-74.
[12] D. Sarjon and Mohd Noor Md Sap, "Association Rules using Rough Set
and Association Rule Methods", Proceedings of 7th Pacific Rim
International Conference on Artificial Intelligence (PRICAI-02), Tokyo,
Japan, August 18-22, 2002, pp. 238-243.
[13] S. Theodoridis, K. Koutroubas, "Pattern recognition", Academic Press,
1999.
[14] S. Albayrak, Fatih Amasyali, Fuzzy c-means clustering on Medical
Diagnostic Systems, International XII. Turkish Symposium on Artificial
Intelligence and Neural Networks, TAINN 2003.
[15] J. Bezkek, "Pattern Recognition with Fuzzy Objective Function
Algorithms", Plenum Press, USA, 1981.
[16] KDD data set, 1999;
http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html
[17] P. Laskov, K. Rieck, C. Schäfer, K.R. Müller, "Visualization of
anomaly detection using prediction sensitivity", Proceeding of
Sicherheit, April 2005, pp. 197-208.
[18] MathWorks, Statistical Toolbox for User-s Guide, MathWorks, 2001.
[19] A. Ôêà hrm, "ROSETTA Technical Reference Manual", Department of
Computer and Information Science, Norwegian University of Science
and Technology (NTNU), Trondheim, Norway, 2000.
[20] W. Chimphlee, Abdul Hanan Abdullah, Mohd Noor Md Sap and S.
Chimphlee, "Unsupervised Anomaly Detection with Unlabeled Data
using Clustering", Proc. Int. Conf. on ICT- Mercu Buana ICT2005. pp.
42-49.
[21] A. Lazarevic, A. Ozgur, L. Ertoz, J. Srivastava, and V. Kumar, "A
comparative study of anomaly detection schemes in network intrusion
detection". In SIAM; International Conference on Data Mining, 2003.
[22] Zhengxin Chen, Data Mining and Uncertain Reasoning - An Integrated
Approach, Wiley, 2001.
[23] Fernando Godínez, Dieter Hutter, Raul Monroy "Attribute Reduction for
Effective Intrusion Detection". AWIC 2004: 74-83.
[24] W. Chimphlee, Abdul Hanan Abdullah, Mohd Noor Md Sap,
S.Chimphlee, and S. Srinoy, Unsupervised Anomaly Detection without
Prior Knowledge Using Clustering, International workshop on
information Technology 2005 (IAIT2005), 25-26 November 2005.,
Thailand.
[25] W. Chimphlee, Mohd Noor Md Sap, Abdul Hanan Abdullah, and S.
Chimphlee, Semi-Supervised Learning to Identify Suspicious Activity
for Anomaly Detection, 3rd International Conference on Computational
Intelligence, Robotics and Autonomous Systems (CIRAS2005), 13-16
December 2005, Singapore.
@article{"International Journal of Information, Control and Computer Sciences:51609", author = "Witcha Chimphlee and Abdul Hanan Abdullah and Mohd Noor Md Sap and Siriporn Chimphlee and Surat Srinoy", title = "Unsupervised Clustering Methods for Identifying Rare Events in Anomaly Detection", abstract = "It is important problems to increase the detection rates
and reduce false positive rates in Intrusion Detection System (IDS).
Although preventative techniques such as access control and
authentication attempt to prevent intruders, these can fail, and as a
second line of defence, intrusion detection has been introduced. Rare
events are events that occur very infrequently, detection of rare
events is a common problem in many domains. In this paper we
propose an intrusion detection method that combines Rough set and
Fuzzy Clustering. Rough set has to decrease the amount of data and
get rid of redundancy. Fuzzy c-means clustering allow objects to
belong to several clusters simultaneously, with different degrees of
membership. Our approach allows us to recognize not only known
attacks but also to detect suspicious activity that may be the result of
a new, unknown attack. The experimental results on Knowledge
Discovery and Data Mining-(KDDCup 1999) Dataset show that the
method is efficient and practical for intrusion detection systems.", keywords = "Network and security, intrusion detection, fuzzy cmeans,rough set.", volume = "1", number = "8", pages = "2351-6", }
