Incorporating Lexical-Semantic Knowledge into Convolutional Neural Network Framework for Pediatric Disease Diagnosis

The utilization of electronic medical record (EMR) data to establish the disease diagnosis model has become an important research content of biomedical informatics. Deep learning can automatically extract features from the massive data, which brings about breakthroughs in the study of EMR data. The challenge is that deep learning lacks semantic knowledge, which leads to impracticability in medical science. This research proposes a method of incorporating lexical-semantic knowledge from abundant entities into a convolutional neural network (CNN) framework for pediatric disease diagnosis. Firstly, medical terms are vectorized into Lexical Semantic Vectors (LSV), which are concatenated with the embedded word vectors of word2vec to enrich the feature representation. Secondly, the semantic distribution of medical terms serves as Semantic Decision Guide (SDG) for the optimization of deep learning models. The study evaluates the performance of LSV-SDG-CNN model on four kinds of Chinese EMR datasets. Additionally, CNN, LSV-CNN, and SDG-CNN are designed as baseline models for comparison. The experimental results show that LSV-SDG-CNN model outperforms baseline models on four kinds of Chinese EMR datasets. The best configuration of the model yielded an F1 score of 86.20%. The results clearly demonstrate that CNN has been effectively guided and optimized by lexical-semantic knowledge, and LSV-SDG-CNN model improves the disease classification accuracy with a clear margin.

• Xiaocong Liu
• Huazhen Wang
• Ting He
• Xiaozheng Li
• Weihan Zhang
• Jian Chen

• lexical semantics
• feature representation
• semantic decision
• convolutional neural network
• electronic medical record

[1] A. Boonstra and M. Broekhuis, “Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions,” BMC Health Services Research, vol. 10, no. 1, pp. 231–241, 2010.
[2] W. Mackinnon and M. Wasserman, “Integrated electronic medical record systems: Critical success factors for implementation,” in Proc. of the Hawaii International Conference on System Sciences, 2009, pp. 1–10.
[3] Y. Li, B. Qian, X. Zhang, et al., “Graph neural network-based diagnosis prediction,” Big Data, vol. 8, no. 5, pp. 379-390, 2020
[4] Y. Li, R Shishir, Solares, et al., “BEHRT: Transformer for electronic health records,” Entific Reports, vol. 10, no. 1, pp. 7155-7167, 2020.
[5] J. Gao, X. Wang, Y. Wang, et al., “CAMP: Co-attention memory networks for diagnosis prediction in healthcare,” in Proc. of the 19th IEEE International Conference on Data Mining, New York, 2019, pp. 1036–1041.
[6] X. S. Hang, F. Tang, H. H. Dodge, et al., “MetaPred: Meta-learning for clinical risk prediction with limited patient electronic health records,” in Proc. of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, 2019, pp. 2487–2495.
[7] W. Wang, H. Xu, Z. Gan, et al., “Graph-driven generative models for heterogeneous multi-task learning” in Proc. of the 35th AAAI Conference on Artificial Intelligence. Menlo Park, 2020, pp. 979–988.
[8] J. Jiang, H. Wang, J. Xie, et al., “Medical knowledge embedding based on recursive neural network for multi-disease diagnosis,” Artificial Intelligence in Medicine, vol. 103, no. 1, pp. 101772–101787, 2020.
[9] L. Wang, H. Wang, Y. Song, et al., “MCPL-Based FT-LSTM: medical representation learning-based clinical prediction model for time series events,” IEEE Access, vol. 7, no. 1, pp. 70253–70264, 2019.
[10] H. Liang, B. Y. Tsui, H. Ni, et al., “Evaluation and accurate diagnoses of pediatric diseases using artificial intelligence,” Nature medicine, vol. 25, no. 3, pp. 433–443, 2019.
[11] A. Bordes, J. Weston, R. Collobert, et al., “Learning structured embeddings of knowledge bases,” in Proc. of the 25th AAAI Conference on Artificial Intelligence, Menlo Park, vol. 25, no. 1, 2011.
[12] A. Bordes, N. Usunier, A. Garcia-Duran, et al., “Translating embeddings for modeling multi-relational data,” in Proc. of the neural information processing systems, Cambridge, 2013, pp. 2787–2795.
[13] A. Rajkomar, E. Oren, C. Kai, et al., “Scalable and accurate deep learning with electronic health records,” NPJ Digital Medicine, vol. 1, no. 1, pp. 18, 2018.
[14] T. Ching, D. S. Himmelstein, B. K. Beaulieu-Jones, et al., “Opportunities and obstacles for deep learning in biology and medicine,” Journal of the Royal Society Interface, vol. 15, no. 141, pp. 20170387, 2018.
[15] L. Fang, Y. Luo, K. Feng, et al., “Knowledge-enhanced ensemble learning for word embeddings,” in Proc. of the World Wide Web Conference, New York, 2019, pp. 427–437.
[16] J. Xu, Z. Zhang, T. Friedman, et al., “A semantic loss function for deep learning with symbolic knowledge,” in Proc. of the 35th International Conference on Machine Learning, Cambridge, 2018, pp. 5502–5511.
[17] E. Choi, M. T. Bahadori, S. Le, et al., “GRAM: Graph-based attention model for healthcare representation learning,” in Proc of the 23th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, 2017, pp. 787–795.
[18] N. Rojas, L. B. Crane, E. Yeager, et al., “Introduction to Linguistics,” Modern Language Journal, vol. 66, no. 4, pp. 445,1998.
[19] X. Shen, “Pediatrics,” 7th ed. Beijing: People’s Health Publishing House, 2013.
[20] V. Jayawardana, D. Lakmal, N. D. Silva, et al., “Deriving a representative vector for ontology classes with instance word vector embeddings,” in Proc. of the Seventh International Conference on Innovative Computing Technology, 2017, pp. 79-84.
[21] T. Mikolov, K. Chen, G. Corrado, et al., “Efficient estimation of word representations in vector space,” in Proc of 7th International Conference on Learning Representations, Stroudsburg, 2013, pp. 1–12.
[22] N. Liu, P. Lu, W. Zhang, et al., “Knowledge-aware deep dual networks for text-based mortality prediction,” in Proc. of 35th International Conference on Data Engineering, 2019, pp. 1406–1417.
[23] H. Wang, Y. Li, S. A. Khan, et al., “Prediction of breast cancer distant recurrence using natural language processing and knowledge-guided convolutional neural network,” Artificial Intelligence in Medicine, vol. 110, p. 101977, 2020.