4-санға «Автоматтандыру және ақпараттық технологиялар», «Азық-түлік инженериясы және биотехнология», сондай-ақ «Химиялық технология» бағыттары бойынша мақалалар қабылдау жабылды.

Приём статей в четвёртый номер по направлениям «Автоматизация и информационные технологии», «Пищевая инженерия и биотехнология», а также «Химическая технология» закрыт.

Submissions for the fourth issue in the fields of Automation and Information Technology, Food Engineering and Biotechnology, as well as Chemical Technology, are now closed.

Preview

Bulletin of Shakarim University. Technical Sciences

Bulletin of Shakarim University. Technical Sciences

Advanced search

SAC-BASED UPLINK POWER CONTROL IN 5G NR: DEEP REINFORCEMENT LEARNING FOR JOINT THROUGHPUT AND ENERGY EFFICIENCY OPTIMIZATION

https://doi.org/10.53360/2788-7995-2026-3(23)-31

Abstract

In this paper, the application of the Soft Actor-Critic (SAC) approach to uplink power control in 5G New Radio networks is investigated. A Gymnasium-compatible simulator for a 10-user uplink cell is constructed using the 3GPP TR 38.901 Urban Macro NLOS path-loss model with stochastic shadowing and Rayleigh fading at 3.5 GHz. The SAC agent observes SINR, transmit power, and path loss for all users and jointly selects continuous transmit powers in the range [0, 30] dBm. The training reward combines throughput (0.50), an energy-efficiency term (0.25), SINR quality (0.15), and fairness (0.10). Three SAC policies are trained for 500,000 timesteps and evaluated on 100 common scenarios against Full Power, Equal Power, and a simplified 3GPP-inspired Fractional Power Control (FPC) baseline. Using the standard network-level energyefficiency metric, SAC achieves 14.866 Mbit/J at a mean transmit power of 9.905 dBm and a mean throughput of 2.540 Mbps. Network energy efficiency is significantly higher than Full Power and simplified FPC, while its difference from Equal Power is not statistically significant after multiplicity correction. A reward-weight sensitivity study confirms that changing objective priorities shifts the learned throughput-energy-efficiency operating point rather than yielding one universally dominant configuration.

About the Authors

S. Aibagarov
Al-Farabi Kazakh National University
Kazakhstan

Serik Aibagarov – Bachelor student, Junior Reseacher, Computer Science department Almaty

71 Al-Farabi Ave., Almaty, Republic of Kazakhstan, 050040



D. Marlambekov
Al-Farabi Kazakh National University
Kazakhstan

Duman Marlambekov – PhD student, Junior Reseacher, Computer Science department

71 Al-Farabi Ave., Almaty, Republic of Kazakhstan, 050040



A. Mukhanbet
Al-Farabi Kazakh National University
Kazakhstan

Aksultan Mukhanbet – PhD student, Researcher, Computer Science department

71 Al-Farabi Ave., Almaty, Republic of Kazakhstan, 050040



References

1. Shen Z. Adaptive resource allocation in multiuser OFDM systems with proportional rate constraints / Z. Shen, J.G. Andrews, B.L. Evans // IEEE Transactions on Wireless Communications. – 2005. – Vol. 4, № 6. – P. 2726-2737. https://doi.org/10.1109/TWC.2005.858010.

2. Tse D. Fundamentals of Wireless Communication / D. Tse, P. Viswanath. – Cambridge: Cambridge University Press, 2005. – 564 p. https://doi.org/10.1017/CBO9780511807213.

3. Tombaz S. Energy- and cost-efficient ultra-high-capacity wireless access / S. Tombaz, A. Västberg, J. Zander // IEEE Wireless Communications. – 2011. – Vol. 18, № 5. – P. 18-24. https://doi.org/10.1109/MWC.2011.6056688.

4. Zander J. Performance of optimum transmitter power control in cellular radio systems / J. Zander // IEEE Transactions on Vehicular Technology. – 1992. – Vol. 41, № 1. – P. 57-62. https://doi.org/10.1109/25.120145.

5. 3GPP TS 38.213. NR; Physical layer procedures for control. Version 17.4.0 [Electronic resource]. – 3rd Generation Partnership Project, 2022. https://www.3gpp.org/dynareport/38213.htm.

6. Holtkamp H. Minimizing base station power consumption / H. Holtkamp, G. Auer, S. Bazzi, H. Haas // IEEE Journal on Selected Areas in Communications. – 2014. – Vol. 32, № 2. – P. 297-306. https://doi.org/10.1109/JSAC.2013.131009.

7. Applications of deep reinforcement learning in communications and networking: A survey / N.C. Luong et al // IEEE Communications Surveys & Tutorials. – 2019. – Vol. 21, № 4. – P. 3133-3174. https://doi.org/10.1109/COMST.2019.2916583.

8. Application of machine learning in wireless networks: Key techniques and open issues / Y. Sun et al // IEEE Communications Surveys & Tutorials. – 2019. – Vol. 21, № 4. – P. 3072-3108. https://doi.org/10.1109/COMST.2019.2924243.

9. Haarnoja T. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor / T. Haarnoja, A. Zhou, P. Abbeel, S. Levine // Proceedings of the 35th International Conference on Machine Learning (ICML 2018). – 2018. – Vol. 80. – P. 1861-1870. https://proceedings.mlr.press/v80/haarnoja18b.html.

10. Cui W. Spatial deep learning for wireless scheduling / W. Cui, K. Shen, W. Yu // IEEE Journal on Selected Areas in Communications. – 2019. – Vol. 37, № 6. – P. 1248-1261. https://doi.org/10.1109/JSAC.2019.2904352.

11. Deep reinforcement learning for radio resource allocation and management in next generation heterogeneous wireless networks: A survey / A. Alwarafy et al // IEEE Open Journal of the Communications Society. – 2022. – Vol. 3. – P. 1967-1986. https://doi.org/10.1109/OJCOMS.2022.3153226.

12. 3GPP TR 38.901. Study on channel model for frequencies from 0.5 to 100 GHz. Version 17.0.0 [Electronic resource]. – 3rd Generation Partnership Project, 2022. https://www.3gpp.org/dynareport/38901.htm.

13. Naparstek O. Deep multi-user reinforcement learning for distributed dynamic spectrum access / O. Naparstek, K. Cohen // IEEE Transactions on Wireless Communications. – 2019. – Vol. 18, № 1. – P. 310-323. https://doi.org/10.1109/TWC.2018.2878578.

14. Ye H. Deep reinforcement learning based resource allocation for V2V communications / H. Ye, G.Y. Li, B.-H.F. Juang // IEEE Transactions on Vehicular Technology. – 2019. – Vol. 68, № 4. – P. 3163-3173. https://doi.org/10.1109/TVT.2019.2897134.

15. Jain R. A quantitative measure of fairness and discrimination for resource allocation in shared computer systems / R. Jain, D.-M. Chiu, W.R. Hawe. – Hudson: Digital Equipment Corporation, 1984. – 38 p. – (DEC Research Report TR-301). https://arxiv.org/abs/cs/9809099.

16. Stable-Baselines3: Reliable reinforcement learning implementations / A. Raffin et al // Journal of Machine Learning Research. – 2021. – Vol. 22, № 268. – P. 1-8. https://jmlr.org/papers/v22/20-1364.html.

17. Sutton R.S. Reinforcement Learning: An Introduction / R.S. Sutton, A.G. Barto. – 2nd ed. – Cambridge, MA: MIT Press, 2018. – 552 p. http://incompleteideas.net/book/the-book-2nd.html.

18. Human-level control through deep reinforcement learning / V. Mnih et al // Nature. – 2015. – Vol. 518, № 7540. – P. 529-533. https://doi.org/10.1038/nature14236.

19. Proximal policy optimization algorithms / J. Schulman et al // arXiv. – 2017. – arXiv:1707.06347. https://doi.org/10.48550/arXiv.1707.06347.

20. Machine learning paradigms for next-generation wireless networks / C. Jiang et al // IEEE Wireless Communications. – 2017. – Vol. 24, № 2. – P. 98-105. https://doi.org/10.1109/MWC.2016.1500356WC.

21. Eskandari M. Network slicing in O-RAN-enabled cell-free massive MIMO: A DRL-based power control / M. Eskandari, M. Rahmani Ghourtani, A.G. Burr // Proceedings of the 2025 IEEE Wireless Communications and Networking Conference (WCNC). – Milan: IEEE, 2025. https://doi.org/10.1109/WCNC61545.2025.10978801.

22. Anbazhagan S. Adaptive power-saving mode control in NB-IoT networks using soft actor-critic reinforcement learning for optimal power management / S. Anbazhagan, R.K. Mugelan // Scientific Reports. – 2025. – Vol. 15, Art. 34618. https://doi.org/10.1038/s41598-025-18214-4.

23. Base station power control strategy in ultra-dense networks via deep reinforcement learning / Q. Chen et al // Physical Communication. – 2025. – Vol. 71, Art. 102655. https://doi.org/10.1016/j.phycom.2025.102655.


Review

For citations:


Aibagarov S., Marlambekov D., Mukhanbet A. SAC-BASED UPLINK POWER CONTROL IN 5G NR: DEEP REINFORCEMENT LEARNING FOR JOINT THROUGHPUT AND ENERGY EFFICIENCY OPTIMIZATION. Bulletin of Shakarim University. Technical Sciences. 2026;(3(23)):331-342. https://doi.org/10.53360/2788-7995-2026-3(23)-31

Views: 16

JATS XML


Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.


ISSN 2788-7995 (Print)
ISSN 3006-0524 (Online)
X