A Rule-based Approach for Anomaly Detection in Subscriber Usage Pattern
In this report we present a rule-based approach to
detect anomalous telephone calls. The method described here uses
subscriber usage CDR (call detail record) data sampled over two
observation periods: study period and test period. The study period
contains call records of customers- non-anomalous behaviour.
Customers are first grouped according to their similar usage
behaviour (like, average number of local calls per week, etc). For
customers in each group, we develop a probabilistic model to describe
their usage. Next, we use maximum likelihood estimation (MLE) to
estimate the parameters of the calling behaviour. Then we determine
thresholds by calculating acceptable change within a group. MLE is
used on the data in the test period to estimate the parameters of the
calling behaviour. These parameters are compared against thresholds.
Any deviation beyond the threshold is used to raise an alarm. This
method has the advantage of identifying local anomalies as compared
to techniques which identify global anomalies. The method is tested
for 90 days of study data and 10 days of test data of telecom
customers. For medium to large deviations in the data in test window,
the method is able to identify 90% of anomalous usage with less than
1% false alarm rate.
[1] CFCA press release, http://www.cfca.org/pdf/press/3-28-06PR.pdf, 2006
(last accessed on October 18, 2007).
[2] Shawe-Taylor, J., Howker, K., Gosset, P., Hyland, M., Verrelst, H.,
Moreau, Y., et al, "Novel techniques for profiling and fraud in mobile
telecommunications," In P. J. G. Lisboa, B. Edisbury, A. Vellido (Eds.),
Business Applications of Neural Networks. The State-of-the-art of Real
World Applications, pp. 113-139, Singapore: World Scientific.
[3] Hoath. P, "What-s new in telecoms fraud?," Computer Fraud and
Security, Vol. 1, pp. 10-14, 1998.
[4] Estevez, P. A, Held, C. M., Perez, C. A., "Subscription Fraud Prevention
in Telecommunications using Fuzzy Rules and Neural Networks," Expert
Systems with Applications, Vol. 31, pp. 337-344, 2006.
[5] Fawcett, T., Provost, F., "Combining Data Mining and Machine Learning
for User Profiling," In AI approaches to fraud detection and risk
management, workshop technical report WS-97-07, pp. 14-19, AAAI
Press, 1997.
[6] Taniguchi, M., Haft, M., Hollmen, J., Tresp, V., "Fraud Detection in
Communication Networks using Neural and Probabilistic Methods,"
IEEE International Conference in Acoustics, Speech and Signal
Processing, Vol. 2, pp. 1241-4, 1998.
[7] Fawcett, T., Provost, F., "Adaptive Fraud Detection," Data Mining and
Knowledge Discovery, Vol. 1, pp. 291-316, 1997.
[8] Hollmen, J., "Novelty Filter for Fraud Detection in Mobile
Communications Networks," Technical Report submitted to Department
of Computer Science and Engineering, Helsinki University of
Technology, 1997.
[9] Burge, P., Shawe-Taylor, J., "Detecting cellular fraud using adaptive
prototypes," AAAI Workshop on AI Approaches to Fraud Detection and
Risk Management. AAAI Press, Menlo Park, CA.
[10] Burge, P., Shawe-Taylor, J., "An Unsupervised Neural Network
Approach to Profiling the Behaviour of Mobile Phone Users for Use in
Fraud Detection," Journal of Parallel and Distributed Computing, Vol.
61, 2001.
[11] Grabec, I, "Modelling of Chaos by a Self-Organizing Neural Network,"
Proceedings of International Conference on Artificial Neural Networks,
Vol. 1, pp. 151-156, Elsevier publications, 1989.
[12] Moreau, Y., Vandewalle, J., "Detection of mobile phone fraud using
supervised neural networks: A first prototype," International Conference
on Artificial Neural Networks, 1065- 1070. Springer, Berlin., 1997.
[13] Moreau, Y., "A hybrid system for fraud detection in mobile
communication," European Symposium on Artificial Neural Networks,
pp. 447-454, 1999.
[14] Cortes, C., Prebigon, D., "Signature-Based Methods for Data Streams,"
Data Mining and Knowledge Discovery, pp. 167-182, 2001.
[15] Cortes, C., Prebigon, D., Volinsky, C., "Communities of Interest,"
Intelligent Data Analysis 2001, pp. 105-114, 2001.
[16] Cortes, C., Prebigon, D., Volinsky, C., "Computational Methods for
Dynamic Graphs," Journal of Computational and Graphical Statistics,
Vol. 12, pp. 950-970, 2003.
[17] Ferreira, et al. "Establishing Fraud Detection Patterns Based on
Signatures," Industrial Conference on Data Mining, LNAI Springer-
Verlag, 2006.
[18] Grosser, et al. "Detecting Fraud in Mobile Telephony Using Neural
Networks," Lecture Notes in AI 3533, Springer-Verlag, 2005
[19] Cahill, M. H., Lambert, D., Pinheiro, J. C., Sun, D. X. "Detecting Fraud
in Real World", In J. Abello, P. M. Pardalos, M. G. C. Resende (Eds.),
Handbook of Massive Datasets, pp. 913-930, Kluwer Academic
Publishers, 2002.
[20] Rosset, S., Neumann, E., Eick, U., Vatnik, N., "Customer LTV modelling
and its use for Customer Retention Planning", Data Mining and
Knowledge Discovery, Vol. 7, pp. 321-339,2003.
[21] Samfat, D., Molva, R., "IDAMN: an intrusion detection architecture for
mobile networks", IEEE Journal on Selected areas of Communications,
Vol. 15, pp. 1373-1380, 1997.
[22] Duda, R. O., Hart, P. E., Stork, D. G., "Pattern Classification", Second
edition, John-Wiley and Sons, 2001.
[1] CFCA press release, http://www.cfca.org/pdf/press/3-28-06PR.pdf, 2006
(last accessed on October 18, 2007).
[2] Shawe-Taylor, J., Howker, K., Gosset, P., Hyland, M., Verrelst, H.,
Moreau, Y., et al, "Novel techniques for profiling and fraud in mobile
telecommunications," In P. J. G. Lisboa, B. Edisbury, A. Vellido (Eds.),
Business Applications of Neural Networks. The State-of-the-art of Real
World Applications, pp. 113-139, Singapore: World Scientific.
[3] Hoath. P, "What-s new in telecoms fraud?," Computer Fraud and
Security, Vol. 1, pp. 10-14, 1998.
[4] Estevez, P. A, Held, C. M., Perez, C. A., "Subscription Fraud Prevention
in Telecommunications using Fuzzy Rules and Neural Networks," Expert
Systems with Applications, Vol. 31, pp. 337-344, 2006.
[5] Fawcett, T., Provost, F., "Combining Data Mining and Machine Learning
for User Profiling," In AI approaches to fraud detection and risk
management, workshop technical report WS-97-07, pp. 14-19, AAAI
Press, 1997.
[6] Taniguchi, M., Haft, M., Hollmen, J., Tresp, V., "Fraud Detection in
Communication Networks using Neural and Probabilistic Methods,"
IEEE International Conference in Acoustics, Speech and Signal
Processing, Vol. 2, pp. 1241-4, 1998.
[7] Fawcett, T., Provost, F., "Adaptive Fraud Detection," Data Mining and
Knowledge Discovery, Vol. 1, pp. 291-316, 1997.
[8] Hollmen, J., "Novelty Filter for Fraud Detection in Mobile
Communications Networks," Technical Report submitted to Department
of Computer Science and Engineering, Helsinki University of
Technology, 1997.
[9] Burge, P., Shawe-Taylor, J., "Detecting cellular fraud using adaptive
prototypes," AAAI Workshop on AI Approaches to Fraud Detection and
Risk Management. AAAI Press, Menlo Park, CA.
[10] Burge, P., Shawe-Taylor, J., "An Unsupervised Neural Network
Approach to Profiling the Behaviour of Mobile Phone Users for Use in
Fraud Detection," Journal of Parallel and Distributed Computing, Vol.
61, 2001.
[11] Grabec, I, "Modelling of Chaos by a Self-Organizing Neural Network,"
Proceedings of International Conference on Artificial Neural Networks,
Vol. 1, pp. 151-156, Elsevier publications, 1989.
[12] Moreau, Y., Vandewalle, J., "Detection of mobile phone fraud using
supervised neural networks: A first prototype," International Conference
on Artificial Neural Networks, 1065- 1070. Springer, Berlin., 1997.
[13] Moreau, Y., "A hybrid system for fraud detection in mobile
communication," European Symposium on Artificial Neural Networks,
pp. 447-454, 1999.
[14] Cortes, C., Prebigon, D., "Signature-Based Methods for Data Streams,"
Data Mining and Knowledge Discovery, pp. 167-182, 2001.
[15] Cortes, C., Prebigon, D., Volinsky, C., "Communities of Interest,"
Intelligent Data Analysis 2001, pp. 105-114, 2001.
[16] Cortes, C., Prebigon, D., Volinsky, C., "Computational Methods for
Dynamic Graphs," Journal of Computational and Graphical Statistics,
Vol. 12, pp. 950-970, 2003.
[17] Ferreira, et al. "Establishing Fraud Detection Patterns Based on
Signatures," Industrial Conference on Data Mining, LNAI Springer-
Verlag, 2006.
[18] Grosser, et al. "Detecting Fraud in Mobile Telephony Using Neural
Networks," Lecture Notes in AI 3533, Springer-Verlag, 2005
[19] Cahill, M. H., Lambert, D., Pinheiro, J. C., Sun, D. X. "Detecting Fraud
in Real World", In J. Abello, P. M. Pardalos, M. G. C. Resende (Eds.),
Handbook of Massive Datasets, pp. 913-930, Kluwer Academic
Publishers, 2002.
[20] Rosset, S., Neumann, E., Eick, U., Vatnik, N., "Customer LTV modelling
and its use for Customer Retention Planning", Data Mining and
Knowledge Discovery, Vol. 7, pp. 321-339,2003.
[21] Samfat, D., Molva, R., "IDAMN: an intrusion detection architecture for
mobile networks", IEEE Journal on Selected areas of Communications,
Vol. 15, pp. 1373-1380, 1997.
[22] Duda, R. O., Hart, P. E., Stork, D. G., "Pattern Classification", Second
edition, John-Wiley and Sons, 2001.
@article{"International Journal of Information, Control and Computer Sciences:61212", author = "Rupesh K. Gopal and Saroj K. Meher", title = "A Rule-based Approach for Anomaly Detection in Subscriber Usage Pattern", abstract = "In this report we present a rule-based approach to
detect anomalous telephone calls. The method described here uses
subscriber usage CDR (call detail record) data sampled over two
observation periods: study period and test period. The study period
contains call records of customers- non-anomalous behaviour.
Customers are first grouped according to their similar usage
behaviour (like, average number of local calls per week, etc). For
customers in each group, we develop a probabilistic model to describe
their usage. Next, we use maximum likelihood estimation (MLE) to
estimate the parameters of the calling behaviour. Then we determine
thresholds by calculating acceptable change within a group. MLE is
used on the data in the test period to estimate the parameters of the
calling behaviour. These parameters are compared against thresholds.
Any deviation beyond the threshold is used to raise an alarm. This
method has the advantage of identifying local anomalies as compared
to techniques which identify global anomalies. The method is tested
for 90 days of study data and 10 days of test data of telecom
customers. For medium to large deviations in the data in test window,
the method is able to identify 90% of anomalous usage with less than
1% false alarm rate.", keywords = "Subscription fraud, fraud detection, anomalydetection, maximum likelihood estimation, rule based systems.", volume = "1", number = "10", pages = "3227-4", }
