Classification of knowledge graph completeness measurement techniques

doi:10.23919/JSEE.2023.000150

Journal of Systems Engineering and Electronics ›› 2024, Vol. 35 ›› Issue (1): 154-162.doi: 10.23919/JSEE.2023.000150

• SYSTEMS ENGINEERING • Previous Articles

Classification of knowledge graph completeness measurement techniques

Ying ZHANG(), Gang XIAO()

¹ Institute of Systems Engineering, Academy of Military Sciences, Beijing 100107, China

Received:2022-12-11 Online:2024-02-18 Published:2024-03-05
Contact: Ying ZHANG E-mail:yingzhang199608@foxmail.com;searchware@qq.com
About author:
ZHANG Ying was born in 1996. She received her M.S. degree in data science from King ’s College of London (KCL), British, in 2020. She is working toward her Ph.D. degree in the Institute of Systems Engineering, Academy of Military Sciences. Her research interests include knowledge graph refinement, and knowledge graph evaluation. E-mail: yingzhang199608@foxmail.com

XIAO Gang was born in 1973. He received his B.S. and Ph.D. degrees in computer science and technology from National University of Defense Technology. He is currently a professor of National Key Laboratory for Complex Systems Simulation. His research interests include complex system theory, system design, knowledge graph, and deep learning. E-mail: searchware@qq.com
Supported by:
This work was supported by the National Key Laboratory for Complex Systems Simulation Foundation (6142006190301).

Abstract

Abstract:

At present, although knowledge graphs have been widely used in various fields such as recommendation systems, question and answer systems, and intelligent search, there are always quality problems such as knowledge omissions and errors. Quality assessment and control, as an important means to ensure the quality of knowledge, can make the applications based on knowledge graphs more complete and more accurate by reasonably assessing the knowledge graphs and fixing and improving the quality problems at the same time. Therefore, as an indispensable part of the knowledge graph construction process, the results of quality assessment and control determine the usefulness of the knowledge graph. Among them, the assessment and enhancement of completeness, as an important part of the assessment and control phase, determine whether the knowledge graph can fully reflect objective phenomena and reveal potential connections among entities. In this paper, we review specific techniques of completeness assessment and classify completeness assessment techniques in terms of closed world assumptions, open world assumptions, and partial completeness assumptions. The purpose of this paper is to further promote the development of knowledge graph quality control and to lay the foundation for subsequent research on the completeness assessment of knowledge graphs by reviewing and classifying completeness assessment techniques.

Key words: quality assessment, completeness assessment, closed world assumptions, open world assumption, partial completeness assumption

Ying ZHANG, Gang XIAO. Classification of knowledge graph completeness measurement techniques[J]. Journal of Systems Engineering and Electronics, 2024, 35(1): 154-162.

Figures/Tables 1

References 43

1	SHETH A, PADHEE S, GYRARD A Knowledge graphs and knowledge networks: the story in brief. IEEE Internet Computing, 2019, 23 (4): 67- 75. doi: 10.1109/MIC.2019.2928449
2	PAULHEIM H Knowledge graph refinement: a survey of approaches and evaluation methods. Semantic Web, 2017, 8 (3): 489- 508.
3	XUE B C, ZOU L Knowledge graph quality management: a comprehensive survey. IEEE Trans. on Knowledge and Data Engineering, 2023, 35 (5): 4969- 4988.
4	NOY N, GAO Y Q, JAIN A, et al Industry-scale knowledge graphs: lessons and challenges: five diverse technology companies show how it’s done. Queue, 2019, 17 (2): 48- 75. doi: 10.1145/3329781.3332266
5	CAO M L, ZHANG J, XU S Y, et al. Knowledge graphs meet crowdsourcing: a brief survey. Proc. of the International Conference on Cloud Computing, 2020: 3−17.
6	SIMSEK U, ANGELE K, KARLE E, et al Knowledge graph lifecycle: building and maintaining knowledge graphs. Proc. of the 2nd International Workshop on Knowledge Graph Construction Co-located with the 18th Extended Semantic Web Conference, 2021, 2873.
7	WANG X Y, CHEN L Z, BAN T Y, et al Knowledge graph quality control: a survey. Fundamental Research, 2021, 1 (5): 607- 626. doi: 10.1016/j.fmre.2021.09.003
8	LEE Y W, STRONG D M, KAHN B K, et al AIMQ: a methodology for information quality assessment. Information & Management, 2002, 40 (2): 133- 146.
9	HOGAN A, GUTIERREZ C, COCHCZ M, et al. Quality Assessment. Cham: Springer International Publishing, 2022.
10	RYEN V, SOYLU A, ROMAN D Building semantic knowledge graphs from (semi-) structured data: a review. Future Internet, 2022, 14 (5): 129. doi: 10.3390/fi14050129
11	ABU-SALIH B Domain-specific knowledge graphs: a survey. Journal of Network and Computer Applications, 2021, 185, 103076.
12	HUAMAN E. Steps to knowledge graphs quality assessment. https://doi.org/10.48550/arXiv.2208.07779.
13	ABIAN D, MERONO-PENUELA A, SIMPERL E An analysis of content gaps versus user needs in the wikidata knowledge graph. Proc. of the International Semantic Web Conference, 2022, 354- 374.
14	GE Z. The future and fintech: ABCDI and beyond. Singapore: World Scientific Publishing Company Inc, 2022.
15	SIMSEK U, KARLE E, ANGELE K, et al A knowledge graph perspective on knowledge engineering. SN Computer Science, 2022, 4 (1): 16. doi: 10.1007/s42979-022-01429-x
16	ZAVERI A, RULA A, MAURINO A, et al Quality assessment for linked data: a survey. Semantic Web, 2016, 7 (1): 63- 93.
17	RANGANATHAN V, BARBOSA D HOPLoP: multi-hop link prediction over knowledge graph embeddings. World Wide Web, 2022, 25 (2): 1037- 1065. doi: 10.1007/s11280-021-00972-6
18	ZHOU Y, CHEN X N, HE B, et al. Re-thinking knowledge graph completion evaluation from an information retrieval perspective. https://arxiv.org/abs/2205.04105.
19	AMSTERDAMER Y, GASPAR L Interactive knowledge graph querying through examples and facets. Proc. of the European Conference on Advances in Databases and Information Systems, 2022, 201- 211.
20	HOGAN A, BLOMQVIST E, COCHEA M, et al Knowledge graphs. ACM Computing Surveys, 2021, 54 (4): 1- 37.
21	FENSEL D, SIMSEK U, ANGELE K, et al. Knowledge graphs. Cham: Springer International Publishing, 2020.
22	ISSA S, ADEKUNLE O, HAMDI F, et al Knowledge graph completeness: a systematic literature review. IEEE Access, 2021, 9, 31322- 31339. doi: 10.1109/ACCESS.2021.3056622
23	MOTRO A Integrity= validity+ completeness. ACM Transaction on Database Systems, 1989, 14 (4): 480- 502. doi: 10.1145/76902.76904
24	DARARI F, NUTT W, RAZNIEWSKI S Comparing index structures for completeness reasoning. Proc. of the IEEE International Workshop on Big Data and Information Security, 2018, 49- 56.
25	GALARRAGA L, RAZNIEWSKI S, AMARILLI A, et al Predicting completeness in knowledge bases. Proc. of the 10th ACM International Conference on Web Search and Data Mining, 2017, 375- 383.
26	PRASOJO R E, DARARI F, RAZNIEWSKI S, et al. Managing and consuming completeness information for wikidata using COOL-WD. Proc. of the 7th International Workshop on Consuming Linked Data, Co-located with the 15th International Semantic Web Conference. https://ceur-ws.org/Vol-1666/paper-02.pdf.
27	DRUMMOND N, SHEARER R. The open world assumption. https://www.cs.man.ac.uk/~drummond/presentations/OWA.pdf.
28	FARBER M, BARTSCHERER F, MENNE C, et al Linked data quality of dbpedia, freebase, opencyc, wikidata, and yago. Semantic Web, 2018, 9 (1): 77- 129.
29	LEVY A Y Obtaining complete answers from incomplete databases. Proc. of the 22nd international Conference on Very Large Data Base Endowment, 1996, 402- 412.
30	RAZNIEWSKI S, NUTT W Completeness of queries over incomplete databases. Proceeding of the Very Large Data Base Endowment, 2011, 4 (11): 749- 760.
31	LAJUS J, SUCHANEK F M Are all people married? Determining obligatory attributes in knowledge bases. Proc. of the World Wide Web Conference, 2018, 1115- 1124.
32	BALARAMAN V, RAZNIEWSKI S, NUTT W Recoin: relative completeness in wikidata. Proc. of the Web Conference, 2018, 1787- 1792.
33	CAPPIELLO C, NOIA T D, MARCU B A, et al A quality model for linked data exploration. Proc. of the International Conference on Web Engineering, 2016, 397- 404.
34	WISESA A, DARARI F, KRISNADHI A, et al Wikidata completeness profiling using proWD. Proc. of the 10th International Conference on Knowledge Capture, 2019, 123- 130.
35	ISSA S, PARIS P H, HAMDI F Assessing the completeness evolution of DBpedia: a case study. Proc. of the International Conference on Conceptual Modeling, 2017, 238- 247.
36	FURBER C, HEPP M Swiqa–a semantic web information quality assessment framework. Proc. of the European Conference on Information Systems, 2011, 76.
37	DARARI F, RAZNIEWSKI S, PRASOJO R et al. Enabling fine-grained RDF data completeness assessment. Proc. of the International Conference on Web Engineering, 2016, 170- 187.
38	LUGGEN M, DIFALLAH D, SARASUA C, et al Non-parametric class completeness estimators for collaborative knowledge graphs—the case of wikidata. Proc. of the International Semantic Web Conference, 2019, 453- 469.
39	CHERIX D, USBECK R, BOTH A, et al. CROCUS: cluster-based ontology data cleansing. https://ceur-ws.org/Vol-1240/wasabi2014-paper1.pdf.
40	SOULET A, GIACOMETTI A, MARKHOFF B, et al Representativeness of knowledge bases with the generalized Benford’s law. Proc. of the International Semantic Web Conference, 2018, 374- 390.
41	NAHARI M K, GHADIRI N, JAFARIFARD Z, et al A framework for linked data fusion and quality assessment. Proc. of the 3rd International Conference on Web Research, 2017, 67- 72.
42	YAGHOUTI N, KAHANI M, BEHKAMAL B. A metric-driven approach for interlinking assessment of RDF graphs. Proc. of the International Symposium on Computer Science and Software Engineering, 2015. DOI: 10.1109/csicsse.2015.7369244.
43	THAKKAR H, ENDRIS K M, GIMENEZ-GARCIA J M, et al. Are linked datasets fit for open-domain question answering? A quality assessment. Proc. of the 6th International Conference on Web Intelligence, Mining and Semantics, 2016. DOI: 10.1145/2912845.2912857.

Completeness metric	Completeness assumption
Completeness metric	Closed world assumption	Open world assumption	Partial completeness assumption
Schema completeness	[28,31−34]	[35]
Property completeness/column completeness	[31,36]	[35]	[25,37]
Population completeness/entity completeness	[31]	[38−40]
Interlinking completeness		[41−43]

Classification of knowledge graph completeness measurement techniques

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

Share this article

Figures/Tables 1

References 43

Related Articles 1

Recommended Articles

Metrics

Comments