INTELLIGENT DEEP LEARNING-BASED HYBRID MODELS FOR MALWARE DETECTION IN THE ICT ENVIRONMENT

This paper presents the research and development of an intelligent hybrid model based on deep learning methods for malware detection in information and communication infrastructures. The goal of the study is to create an architecture combining convolutional (CNN), recurrent (GRU), and graph (GNN) neural networks with ontology-based feature normalization. This approach provides a comprehensive analysis of the static, behavioral, and structural characteristics of malware, increasing resilience to zero-day attacks and polymorphic threats. The EMBER-2018, Malimg, CICAndMal2017, and CICMalDroid-2020 open datasets were used for experimental validation. The experiments showed that the proposed hybrid architecture provides a classification accuracy of 98.7% and a ROC-AUC of 0.99, outperforming isolated deep learning models and traditional machine analysis methods. The introduction of an ontological knowledge model increased the system's interpretability and reduced the false-positive rate to 1.7%. The developed software prototype demonstrated the solution's applicability in real-time systems (SOC/IDS) and demonstrated potential for further scaling and implementation in practical cybersecurity systems. The scientific novelty of the study lies in the integration of multimodal analysis and ontological modeling, which enables more accurate and explainable detection of malicious objects in complex digital environments.

1. CHislo – goda resheniya laboratorii Laboratoriya Kasperskogo ezhednevno obnaruzhivayut 467 tysyach novyh vredonosnyh fajlov // Press-reliz. – Moskovskaya obl., 4 dek. 2024. https://www.kaspersky.ru/about/press-releases/chislo-goda-resheniya-laboratorii-kasperskogoezhednevno-obnaruzhivayut-467-tysyach-novyh-vredonosnyh-fajlov.

2. Malware visualization for deep learning detection / Q. Cui et al // IEEE Access. – 2022. – Т. 10. – Р. 24435-24447.

3. Saxe J. Deep Neural Network Based Malware Detection Using Two-Dimensional Binary Program Features / J. Saxe, K. Berlin // Proc. 10th Int. Conf. on Malicious and Unwanted Software (MALWARE). – 2015. – Р. 11-20. https://doi.org/10.1109/MALWARE.2015.7413680.

4. Hybrid static and dynamic analysis for malware detection / A. Marastoni et al // Computers & Security. – 2024. – Vol. 135. – Article 109934.

5. AV-Comparatives. Real-World Protection Test 2024 – Summary Report // AVComparatives.org. – 2024.

6. Anderson H.S. EMBER: An open dataset for training static PE malware machine learning models / H.S. Anderson, P. Roth // arXiv. – 2018. – arXiv:1804.04637.

7. Malware Images: Visualization and Automatic Classification / L. Nataraj et al // Proc. International Symposium on Visualization for Cyber Security (VizSec). – Jul 2011. – P. 1-7. https://doi.org/10.1145/2016904.2016908.

8. Poornima P. Automated malware detection using machine learning and deep learning approaches for android applications / P. Poornima, G. Mahalakshmi // Expert Systems with Applications. – 2024. – Т. 237. – Article 121456.

9. Graph Neural Networks for Malware Detection: A Survey / Y. Zhu et al // IEEE Access. – 2023. – Т. 11. – Р. 85123-85147.

10. Ding Y. Ontology-based knowledge representation for malware individuals and families / Y. Ding, R. Wu, X. Xiao // Computers & Security. – 2019. – Vol. 87. – Art. 101574. https://doi.org/10.1016/j.cose.2019.101574.

11. Deldar F. Deep Learning for Zero-day Malware Detection and Classification: A Survey / F. Deldar, M. Abadi // ACM Computing Surveys. – 2023. – Vol. 56, № 2. – P. 1-37. https://doi.org/10.1145/3605775.

12. ScaleMalNet: A Scalable Hybrid Deep Learning Model for Malware Classification / R. Vinayakumar et al // Future Generation Computer Systems. – 2020. – Т. 115. – P. 280-292.

13. Alshoulie M. Deep Learning Approaches for Malware Detection: A Comprehensive Review of Techniques, Challenges, and Future Directions / M. Alshoulie, A. Mehmood // IEEE Access. – 2025. – Т. 13. – P. 118652-118677. https://doi.org/10.1109/ACCESS.2025.3582875.

14. Machine learning based fileless malware traffic classification using image visualization / F.A. Demmese et al // Cybersecurity. – 2023. – Т. 6, № 1. – Art. 32. https://doi.org/10.1186/s42400-023-00170-z.

15. A framework for detecting zero-day exploits in network flows / A. Touré et al // Computer Networks. – 2024. – Vol. 224. – Art. 110476. https://doi.org/10.1016/j.comnet.2024.110476.

16. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization / R.R. Selvaraju et al // Proceedings of the IEEE International Conference on Computer Vision (ICCV). – 2017. – P. 618-626. https://doi.org/10.1109/ICCV.2017.74. https://doi.org/10.48550/arXiv.1610.02391

17. GNNExplainer: Generating Explanations for Graph Neural Networks / R. Ying et al // Advances in Neural Information Processing Systems (NeurIPS). – 2019. – Vol. 32. – P. 9240-9251. https://doi.org/10.48550/arXiv.1903.03894.

18. Abduraimova B. Comparative study of machine learning applications in malware forensics / B. Abduraimova, S. Gnatyuk, A. Nurmukhanbetova // Proceedings of the Workshop on Cybersecurity Providing in Information and Telecommunication Systems II (CPITS-II 2024). – Kyiv, Ukraine, October 26, 2024. – CEUR Workshop Proceedings, Vol. 3826. – P. 139-152. https://ceur-ws.org/Vol3826/paper13.pdf (date of request: 03.11.2025).

19. AlOmari H. A comparative analysis of machine learning algorithms for Android malware detection / H. AlOmari, Q.M. Yaseen, M.A. Al-Betar // Procedia Computer Science. – 2023. – Vol. 220. – P. 763-768. https://doi.org/10.1016/j.procs.2023.03.101.

20. Analyzing and comparing the effectiveness of malware detection: A study of machine learning approaches / M. Azeem et al // Heliyon Computer Science. – 2024. – Vol. 10, № 1. – Art. e24058. https://www.sciencedirect.com/science/article/pii/S2405844023107821.

21. A robust machine learning-based mechanism for malware attack detection and analysis / R. Bakshi et al // Procedia Computer Science. – 2025. – Vol. 242. – P. 876-885. https://www.sciencedirect.com/science/article/pii/S1877050925010646.

22. Efficient malware detection using NLP and deep learning model / U. Gupta et al // Alexandria Engineering Journal. – 2025. – Vol. 124. – P. 550-564. https://www.sciencedirect.com/science/article/pii/S1110016825004260.