AN AI-DRIVEN THREE-STAGE FRAMEWORK FOR DYNAMIC INTEGRATION OF EMERGING CONCEPTS INTO ONTOLOGIES

This article examines the problem of automatic implementation of new concepts in ontology in the context of constant updating of knowledge and increasing the volume of textual data. The object of research is the process by which new concepts can be integrated into hierarchical and non-taxonomic structures of ontology, while maintaining their logical and semantic consistency. The purpose of the work is to create a three-stage structure to automate the search, refinement, and optimal implementation of ideas. This structure should be built on the basis of modern machine learning and natural language processing technologies.
Logical inference, contrast learning and large language models and pre-prepared language models are used as research methods. At the initial stage of the framework, larger language models are used to create formal OWL axioms and semantic connections, taking into account universal and existential logical constraints. In the second stage, contrast learning is used to refine vector representations of ideas and improve classification accuracy. The third stage is the logical verification and improvement of non-taxonomic relations with the help of ontological reasoners and external sources of knowledge.
The biomedical ontology of SNOMED CT and the corpus of Kazakh data were used for experimental evaluation. The results show significant statistical superiority over existing methods, as well as high accuracy of concept placement (up to 91%). The main scientific value of the work lies in the fact that it integrates logical constraints and contextual analysis into the task of ontological expansion. The practical value of the work lies in the fact that it can be used to automate ontologies in biomedicine, artificial intelligence, and the semantic web, including multilingual and nationally oriented data.

1. A basic description logic / F. Baader et al // In Cambridge University Press, Cambridge. – 2017. – Р. 10-49. https://doi.org/10.1017/9781139025355.002.

2. Ontology languages and applications / F. Baader et al // In Cambridge University Press, Cambridge. – 2017. – Р. 205-227. https://doi.org/10.1017/9781139025355.008.

3. Knowledge graphs for the life sciences: Recent developments, challenges, and opportunities / J. Chen et al // ArXiv preprint. – 2023. – arXiv:2309. 17255.

4. Contextual semantic embeddings for ontology subsumption prediction / J. Chen et al // World Wide Web. – 2023. – Р. 1-23.

5. Scaling instruction-finetuned language models / H.W. Chung et al // ArXiv preprint. – 2022. – arXiv:2210. 11416.

6. Qlora: Efficient fine-tuning of quantized llms / T. Dettmers et al // ArXiv preprint. – 2023. – arXiv:2305. 14314.

7. Ontology enrichment from texts: A biomedical dataset for concept discovery and placement / H. Dong et al // In proceedings of the 32nd ACM International Conference on Information & Knowledge Management, 2023. New York, NY, USA. https://doi.org/10.1145/3583780.3615126.

8. Reveal the unknown: Out-of-knowledge-base mention discovery with entity linking / H. Dong et al // Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. – 2023. – Р. 452-462. CIKM '23, New York, NY, USA. https://doi.org/10.1145/3583780.3615036.

9. Retrieval-augmented Generation for large Language models A survey / Y. Gao et al // ArXiv preprint. – 2023. – arXiv:2312. 10997.

10. Gibaja, E. A tutorial on multilabel learning / E. Gibaja, S. Ventura // ACM Comput. Surv. – 2015. – № 47(3). https://doi.org/10.1145/2716262.

11. Interpretable ontology extension in chemistry / M. Glauer et al // Semantic Web Pre-press. – 2023. – Р. 1-22.

12. OWL 2: The next step for owl / B.C. Grau et al // Journal of Web Semantics. – 2008. – № 6(4). – Р. 309-322.

13. Domain-specific language model pretraining for biomedical natural language processing / Y. Gu et al // ACM Trans. Comput. Healthcare. – 2021. – № 3(1). https://doi.org/10.1145/3458754.

14. Exploring large language models for ontology alignment / Y. He et al // ArXiv preprint. – 2023. – arXiv:2309.07172.

15. Deeponto: A python package for ontology engineering with deep learning / Y. He et al // ArXiv preprint. – 2023. – arXiv:2307. 03067.

16. Language model analysis for ontology subsumption inference / Y. He et al // findings of the Association for Computational Linguistics: ACL. – 2023. – Р. 3439-3453. https://doi.org/10.18653/v1/2023.findings-acl.213.

17. Hertling S. Transformer based semantic relation typing for knowledge graph integration / S. Hertling, H. Paulheim // In European Semantic Web Conference. – Р. 105-121. Springer.

18. Jurafsky D. Speech and Language Processing / D. Jurafsky, J.H. Martin // 3rd Edition Online. – 2023.

19. Kudo T. SentencePiece: A simple and language-independent subword tokenizer and detokenizer for neural text processing / T. Kudo, J. Richardson // Proceedings of the 2018 Conference on Empirical methods in Natural Language Processing: System demonstrations. – 2018. – Р. 66-71, Brussels, Belgium. https://doi.org/10.18653/v1/D18-2012.

20. Self-alignment pretraining for biomedical entity representations / F. Liu et al // Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics Human Language Technologies. – 2021. – Р. 4228-4238. https://doi.org/10.18653/v1/2021.naacl.

21. Liu H. Concept placement using Bert trained by transforming and summarizing biomedical ontology structure / H. Liu, Y. Perl, J. Geller // J. of Biomedical Informatics. – 2020. – Р. 112(C).

22. Reimers N. Sentence-BERT: Sentence embeddings using Siamese BERT-networks / N. Reimers, I. Gurevych // Proceedings of the 2019 Conference on Empirical methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). – 2019. – Р. 3982-3992. https://doi.org/10.18653/v1/D19-1410.

23. Ruas P. Nilinker: Attention-based approach to nil entity linking / P. Ruas, F.M. Couto // Journal of Biomedical Informatics. – 2022. – Р. 104137. https://doi.org/10.1016/j.jbi.2022.104137.