The Model of Knowledge Graph Embedding with Text and Relation Path

XIAO Bao; WEI Lina; LI Pu; JIANG Yuncheng

doi:10.6054/j.jscnun.2020101

Journal of South China Normal University (Natural Science Edition) > 2020 > 52(6): 103-112. > DOI: 10.6054/j.jscnun.2020101

XIAO Bao, WEI Lina, LI Pu, JIANG Yuncheng. The Model of Knowledge Graph Embedding with Text and Relation Path[J]. Journal of South China Normal University (Natural Science Edition), 2020, 52(6): 103-112. DOI: 10.6054/j.jscnun.2020101

Citation:

PDF (1239 KB)

The Model of Knowledge Graph Embedding with Text and Relation Path

XIAO Bao^1, ,,
WEI Lina^{2, 4},
LI Pu³,
JIANG Yuncheng²

1.
School of Electronics and Information Engineering, Beibu Gulf University, Qinzhou 535011, China
2.
School of Computer Science, South China Normal University, Guangzhou 510631, China
3.
Software Engineering College, Zhengzhou University of Light Industry, Zhengzhou 450000, China
4.
School of Information Science and Engineering, Guangxi University for Nationalities, Nanning 530006, China

More Information

Received Date: February 24, 2020
Available Online: January 04, 2021

Graphical Abstract

Abstract

Abstract

Considering that the existing models cannot completely take advantage of the semantic information of texts and paths, a new model of knowledge graph embedding (named GETR model) is proposed. First, LDA is used to enrich the semantics of an entity description text and TWE is used to obtain word embedding and topic embedding. To enhance the representation of entities, the modified Bi-LSTM model is exploited to encode word embedding and topic embedding. Furthermore, the multiple-step path between two entities is obtained through random walks with the strategy of combining PageRank and Cosine similarity. Additionally, to filter the noise and improve the efficiency of the model, the important semantics of the multi-step path to be used for joint training with the translation model is captured with the self-attention mechanism. Finally, the proposed model GETR, as well as the baseline models TransE, DKRL and TKGE, is evaluated in the tasks of knowledge graph completion and entity classification with three datasets: FB15K, FB20K, and WN18. Experimental results demonstrate that the proposed model outperforms the baseline models, indicating that the new model is more effective for knowledge representation.
- knowledge graph embedding,
- random walks,
- self-attention mechanism,
- multiple-step path,
- entity description text

FullText(HTML)

References (31)

References

[1]	KALLOUBI F, NFAOUI E H, BEQQALI O E. Microblog semantic context retrieval system based on linked open data and graph-based theory[J]. Expert Systems with Application, 2016, 53:138-148.
[2]	LOPEZ V, UNGER C, CIMIANO P, et al. Evaluating question answering over linked data[J]. Journal of Web Semantics, 2013, 21:3-13.
[3]	贾中浩, 古天龙, 宾辰忠, 等.旅游知识图谱特征学习的景点推荐[J].智能系统学报, 2019, 14(3):430-437. JIA Z H, GU T L, BIN C Z, et al. Tourism knowledge-graph feature learning for attraction recommendations[J]. CAAI Transactions on Intelligent Systems, 2019, 14(3):430-437.
[4]	刘知远, 孙茂松, 林衍凯, 等.知识表示学习研究进展[J].计算机研究与发展, 2016, 53(2):247-261. LIU Z Y, SUN M S, LIN Y K, et al. Knowledge representation learning:a review[J]. Journal of Computer Research and Development, 2016, 53(2):247-261.
[5]	BORDES A, USUNIER N, GARCIADURAN A, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of Neural Information Proccessings Systems. Cambridge, MA: MIT Press, 2013: 2787-2795.
[6]	WANG Z, ZHANG J W, FENG J L, et al. Knowledge graph embedding by translating on hyperplanes[C]//Procee-dings of the Twenty-Eighth AAAI Conference on Artificial Intelligence. Menlo Park, CA: AAI Press, 2014: 1112-1119.
[7]	LIN Y, LIU Z, SUN M, et al. Learning entity and relation embeddings for knowledge graph completion[C]//Proceedings of National Conference on Artificial Intelligence. Menlo Park, CA: AAAI Press, 2015: 2181-2187.
[8]	JI G, HE S, XU L, et al. Knowledge graph embedding via dynamic mapping matrix[C]//Proceedings of the 53rd Annusl Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Na-tural Language Processing.Stroudsburg, PA: ACL, 2015: 687-696.
[9]	LIN Y, LIU Z, LUAN H, et al. Modeling relation paths for representation learning of knowledge bases[C]//Procee-dings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal: ACL, 2015: 705-714.
[10]	AN B, CHEN B, HAN X P, et al. Accurate text-enhanced knowledge graph representation learning[C]//Procee-dings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New Orleans, Louisiana: ACL, 2018: 745-755.
[11]	XIE R B, LIU Z Y, JIA J, et al. Representation learning of knowledge graphs with entity descriptions[C]//Procee-dings of the Thirtieth AAAI Conference on Artificial Intelligence. Menlo Park, CA: AAAI, 2016: 2659-2665.
[12]	TU C C, LIU H, LIU Z Y, et al. Cane: context-aware network embedding for relation modeling[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Vancouver, Canada: ACL, 2017: 1722-1731.
[13]	NIE B, SUN S. Knowledge graph embedding via reasoning over entities, relations, and text[J]. Future Generation Computer Systems, 2019, 91:426-433.
[14]	GUAN N, SONG D, LIAO L, et al. Knowledge graph embedding with concepts[J]. Knowledge Based Systems, 2019, 164:38-44.
[15]	WANG Z, ZHANG J W, FENG J L, et al. Knowledge graph and text jointly embedding[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Doha, Qatar: ACL, 2014: 1591-1601.
[16]	ZHONG H P, ZHANG J W, WANG Z, et al. Aligning knowledge and text embeddings by entity descriptions[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2015: 267-272.
[17]	LECUN Y, BOSER B, DENKER J, et al. Backpropagation applied to handwritten zip code recognition[J]. Neural Computation, 1989, 1(4):541-551.
[18]	MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc, 2013: 3111-3119.
[19]	FAN M, ZHOU Q, ZHENG T F, et al. Distributed representation learning for knowledge graphs with entity descriptions[J]. Pattern Recognition Letters, 2017, 93:31-37.
[20]	LAO N, COHEN W W. Relational retrieval using a combination of path-constrained random walks[J]. Machine Learning, 2010, 81(1):53-67.
[21]	GRAVES A, SCHMIDHUBER J. Framewise phoneme cla-ssification with bidirectional LSTM and other neural network architectures[J]. Neural Networks, 2005, 18(5):602-610.
[22]	ZHOU P, SHI W, TIAN J, et al. Attention-based bidirectional long short-term memory networks for relation cla-ssification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin: ACL, 2016: 207-212.
[23]	PEROZZI B, ALRFOU R, SKIENA S, et al. DeepWalk: online learning of social representations[C]//Procee-dings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2014: 701-710.
[24]	LIU B. Web data mining:exploring hyperlinks, contents, and usage data[M]. New York:Springer, 2007.
[25]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Processings of the 31th Conference on Neural Information Processing Systems. Long Beach, CA: NIPS, 2017: 5998-6008.
[26]	BLEI D M, NG A Y, JORDAN M I, et al. Latent dirichlet allocation[J]. Journal of Machine Learning Research, 2003, 3:993-1022.
[27]	LIU Y, LIU Z, CHUA T S, et al. Topical word embeddings[C]//Proceedings of the 29th AAAI Conference on Artificial Intelligence. Austin, Texas: AAAI Press, 2015: 2418-2424.
[28]	VO D T, OCK C Y. Learning to classify short text from scientific documents using topic models with various types of knowledge[J]. Expert Systems with Applications, 2015, 42(3):1684-1698.
[29]	BA J L, KIROS J R, HINTON G E. Layer normalization[J]. arXiv, (2016-07-21)[2020-01-28]. http://ar-xiv.org/abs/1607.06450.
[30]	SAIF A, AZIZ M J, OMAR N, et al. Reducing explicit semantic representation vectors using Latent Dirichlet Allocation[J]. Knowledge Based Systems, 2016, 100:145-159.
[31]	HE K M, ZHANG X Y, REN S Q, et al. Delving deep into rectifiers: surpassing human-level performance on imagenet classification[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Washington: IEEE, 2015: 1026-1034.

Cited By

Cited by

Periodical cited type(0)

Other cited types(3)

Get Citation

PDF

XML

Article views (1265) PDF downloads (107) Cited by(3)

Turn off MathJax

Article Contents

Abstract

References

The Model of Knowledge Graph Embedding with Text and Relation Path

Abstract

References

Cited by

Periodical cited type(0)

Other cited types(3)

Catalog

Export File

Citation

Format

Content