Multimedia Data Fusion for Event Detection in Twitter by Using Dempster-Shafer Evidence Theory

Data fusion technology can be the best way to extract
useful information from multiple sources of data. It has been widely
applied in various applications. This paper presents a data fusion
approach in multimedia data for event detection in twitter by using
Dempster-Shafer evidence theory. The methodology applies a mining
algorithm to detect the event. There are two types of data in the
fusion. The first is features extracted from text by using the bag-ofwords
method which is calculated using the term frequency-inverse
document frequency (TF-IDF). The second is the visual features
extracted by applying scale-invariant feature transform (SIFT). The
Dempster - Shafer theory of evidence is applied in order to fuse the
information from these two sources. Our experiments have indicated
that comparing to the approaches using individual data source, the
proposed data fusion approach can increase the prediction accuracy
for event detection. The experimental result showed that the proposed
method achieved a high accuracy of 0.97, comparing with 0.93 with
texts only, and 0.86 with images only.

Authors:

• Samar M. Alqhtani
• Suhuai Luo
• Brian Regan

Keywords:

• Data fusion
• Dempster-Shafer theory
• data mining
• event detection.

References:

[1] Z. Xie and L. Guan, "Multimodal information fusion of audiovisual
emotion recognition using novel information theoretic tools," in IEEE
International Conference on Multimedia and Expo (ICME), 2013, pp. 1-
6.
[2] S. Zhou, H. Leung, and F. Yao, "Multimedia Data Fusion,"
Mathematical Problems in Engineering, vol. 2013, 2013.
[3] L. A. Klein, Sensor and data fusion: a tool for information assessment
and decision making vol. 324: SPIE press Bellingham^ eWA WA, 2004.
[4] Mark Maybury and S. Walter, "Multimedia Information Extraction,"
2008.
[5] P. K. Atrey, M. A. Hossain, A. El Saddik, and M. S. Kankanhalli,
"Multimodal fusion for multimedia analysis: a survey," Multimedia
systems, vol. 16, pp. 345-379, 2010.
[6] P. Sharma and M. Kaur, "Multimodal classification using feature level
fusion and SVM," Int. J. Comput. Appl, vol. 76, pp. 26-32, 2013.
[7] J.-P. Isson and J. Harriott, Win with advanced business analytics:
creating business value from your data: John Wiley & Sons, 2012.
[8] N. Mishra and D. S. Silakari, "Image mining in the context of content
based image retrieval: a perspective," IJCSI International Journal of
Computer Science Issues, vol. 9, pp. 98-107, 2012.
[9] Y. Kompatsiaris and P. Hobson, Semantic Multimedia and Ontologies:
Springer, 2008.
[10] A. Ma, A. Flenner, D. Needell, and A. G. Percus, "Improving Image
Clustering using Sparse Text and the Wisdom of the Crowds," arXiv
preprint arXiv:1405.2102, 2014.
[11] S. M. Alqhtani, S. Luo, and B. Regan, "Fusing Text and Image for Event
Detection in Twitter," arXiv preprint arXiv:1503.03920, 2015.
[12] J. Mao, Multimodal Data Fusion As a Predictior of Missing Information
in Social Networks: Arizona State University, 2012.
[13] F. Atefeh and W. Khreich, "A survey of techniques for event detection
in Twitter," Computational Intelligence, vol. Volume 31, 2013.
[14] G. Shafer, "A Mathematical Theory of Evidence," AMC, vol. 10, p. 12,
1976.
[15] J. Li, S. Luo, and J. S. Jin, "Sensor data fusion for accurate cloud
presence prediction using Dempster-Shafer evidence theory," Sensors,
vol. 10, pp. 9384-9396, 2010.
[16] Z. Zhang, T. Liu, and W. Zhang, "Novel Paradigm for Constructing
Masses in Dempster-Shafer Evidence Theory for Wireless Sensor
Network’s Multisource Data Fusion," Sensors, vol. 14, pp. 7049-7065,
2014.
[17] J. G. Li, C. N. Zheng, H. W. Xuan, and Y. Jiang, "Data Fusion in
Environment Monitoring Systems with Extended Dempster-Shafer
Theory," Applied Mechanics and Materials, vol. 543, pp. 1074-1077,
2014.
[18] O. Ozdikis, H. Oguztuzun, and P. Karagoz, "Evidential location
estimation for events detected in twitter," in Proceedings of the 7th
Workshop on Geographic Information Retrieval, 2013, pp. 9-16.
[19] G. Salton and C. Buckley, "Term-weighting approaches in automatic
text retrieval," Information processing & management, vol. 24, pp. 513-
523, 1988.
[20] S. Wold, K. Esbensen, and P. Geladi, "Principal component analysis,"
Chemometrics and intelligent laboratory systems, vol. 2, pp. 37-52,
1987.
[21] D. G. Lowe, "Distinctive image features from scale-invariant keypoints,"
International journal of computer vision, vol. 60, pp. 91-110, 2004.
[22] R. M. Gray, "Vector quantization," ASSP Magazine, IEEE, vol. 1, pp. 4-
29, 1984.