SAC-BASED UPLINK POWER CONTROL IN 5G NR: DEEP REINFORCEMENT LEARNING FOR JOINT THROUGHPUT AND ENERGY EFFICIENCY OPTIMIZATION
https://doi.org/10.53360/2788-7995-2026-3(23)-31
Abstract
In this paper, the application of the Soft Actor-Critic (SAC) approach to uplink power control in 5G New Radio networks is investigated. A Gymnasium-compatible simulator for a 10-user uplink cell is constructed using the 3GPP TR 38.901 Urban Macro NLOS path-loss model with stochastic shadowing and Rayleigh fading at 3.5 GHz. The SAC agent observes SINR, transmit power, and path loss for all users and jointly selects continuous transmit powers in the range [0, 30] dBm. The training reward combines throughput (0.50), an energy-efficiency term (0.25), SINR quality (0.15), and fairness (0.10). Three SAC policies are trained for 500,000 timesteps and evaluated on 100 common scenarios against Full Power, Equal Power, and a simplified 3GPP-inspired Fractional Power Control (FPC) baseline. Using the standard network-level energyefficiency metric, SAC achieves 14.866 Mbit/J at a mean transmit power of 9.905 dBm and a mean throughput of 2.540 Mbps. Network energy efficiency is significantly higher than Full Power and simplified FPC, while its difference from Equal Power is not statistically significant after multiplicity correction. A reward-weight sensitivity study confirms that changing objective priorities shifts the learned throughput-energy-efficiency operating point rather than yielding one universally dominant configuration.
Keywords
About the Authors
S. AibagarovKazakhstan
Serik Aibagarov – Bachelor student, Junior Reseacher, Computer Science department Almaty
71 Al-Farabi Ave., Almaty, Republic of Kazakhstan, 050040
D. Marlambekov
Kazakhstan
Duman Marlambekov – PhD student, Junior Reseacher, Computer Science department
71 Al-Farabi Ave., Almaty, Republic of Kazakhstan, 050040
A. Mukhanbet
Kazakhstan
Aksultan Mukhanbet – PhD student, Researcher, Computer Science department
71 Al-Farabi Ave., Almaty, Republic of Kazakhstan, 050040
References
1. Shen Z. Adaptive resource allocation in multiuser OFDM systems with proportional rate constraints / Z. Shen, J.G. Andrews, B.L. Evans // IEEE Transactions on Wireless Communications. – 2005. – Vol. 4, № 6. – P. 2726-2737. https://doi.org/10.1109/TWC.2005.858010.
2. Tse D. Fundamentals of Wireless Communication / D. Tse, P. Viswanath. – Cambridge: Cambridge University Press, 2005. – 564 p. https://doi.org/10.1017/CBO9780511807213.
3. Tombaz S. Energy- and cost-efficient ultra-high-capacity wireless access / S. Tombaz, A. Västberg, J. Zander // IEEE Wireless Communications. – 2011. – Vol. 18, № 5. – P. 18-24. https://doi.org/10.1109/MWC.2011.6056688.
4. Zander J. Performance of optimum transmitter power control in cellular radio systems / J. Zander // IEEE Transactions on Vehicular Technology. – 1992. – Vol. 41, № 1. – P. 57-62. https://doi.org/10.1109/25.120145.
5. 3GPP TS 38.213. NR; Physical layer procedures for control. Version 17.4.0 [Electronic resource]. – 3rd Generation Partnership Project, 2022. https://www.3gpp.org/dynareport/38213.htm.
6. Holtkamp H. Minimizing base station power consumption / H. Holtkamp, G. Auer, S. Bazzi, H. Haas // IEEE Journal on Selected Areas in Communications. – 2014. – Vol. 32, № 2. – P. 297-306. https://doi.org/10.1109/JSAC.2013.131009.
7. Applications of deep reinforcement learning in communications and networking: A survey / N.C. Luong et al // IEEE Communications Surveys & Tutorials. – 2019. – Vol. 21, № 4. – P. 3133-3174. https://doi.org/10.1109/COMST.2019.2916583.
8. Application of machine learning in wireless networks: Key techniques and open issues / Y. Sun et al // IEEE Communications Surveys & Tutorials. – 2019. – Vol. 21, № 4. – P. 3072-3108. https://doi.org/10.1109/COMST.2019.2924243.
9. Haarnoja T. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor / T. Haarnoja, A. Zhou, P. Abbeel, S. Levine // Proceedings of the 35th International Conference on Machine Learning (ICML 2018). – 2018. – Vol. 80. – P. 1861-1870. https://proceedings.mlr.press/v80/haarnoja18b.html.
10. Cui W. Spatial deep learning for wireless scheduling / W. Cui, K. Shen, W. Yu // IEEE Journal on Selected Areas in Communications. – 2019. – Vol. 37, № 6. – P. 1248-1261. https://doi.org/10.1109/JSAC.2019.2904352.
11. Deep reinforcement learning for radio resource allocation and management in next generation heterogeneous wireless networks: A survey / A. Alwarafy et al // IEEE Open Journal of the Communications Society. – 2022. – Vol. 3. – P. 1967-1986. https://doi.org/10.1109/OJCOMS.2022.3153226.
12. 3GPP TR 38.901. Study on channel model for frequencies from 0.5 to 100 GHz. Version 17.0.0 [Electronic resource]. – 3rd Generation Partnership Project, 2022. https://www.3gpp.org/dynareport/38901.htm.
13. Naparstek O. Deep multi-user reinforcement learning for distributed dynamic spectrum access / O. Naparstek, K. Cohen // IEEE Transactions on Wireless Communications. – 2019. – Vol. 18, № 1. – P. 310-323. https://doi.org/10.1109/TWC.2018.2878578.
14. Ye H. Deep reinforcement learning based resource allocation for V2V communications / H. Ye, G.Y. Li, B.-H.F. Juang // IEEE Transactions on Vehicular Technology. – 2019. – Vol. 68, № 4. – P. 3163-3173. https://doi.org/10.1109/TVT.2019.2897134.
15. Jain R. A quantitative measure of fairness and discrimination for resource allocation in shared computer systems / R. Jain, D.-M. Chiu, W.R. Hawe. – Hudson: Digital Equipment Corporation, 1984. – 38 p. – (DEC Research Report TR-301). https://arxiv.org/abs/cs/9809099.
16. Stable-Baselines3: Reliable reinforcement learning implementations / A. Raffin et al // Journal of Machine Learning Research. – 2021. – Vol. 22, № 268. – P. 1-8. https://jmlr.org/papers/v22/20-1364.html.
17. Sutton R.S. Reinforcement Learning: An Introduction / R.S. Sutton, A.G. Barto. – 2nd ed. – Cambridge, MA: MIT Press, 2018. – 552 p. http://incompleteideas.net/book/the-book-2nd.html.
18. Human-level control through deep reinforcement learning / V. Mnih et al // Nature. – 2015. – Vol. 518, № 7540. – P. 529-533. https://doi.org/10.1038/nature14236.
19. Proximal policy optimization algorithms / J. Schulman et al // arXiv. – 2017. – arXiv:1707.06347. https://doi.org/10.48550/arXiv.1707.06347.
20. Machine learning paradigms for next-generation wireless networks / C. Jiang et al // IEEE Wireless Communications. – 2017. – Vol. 24, № 2. – P. 98-105. https://doi.org/10.1109/MWC.2016.1500356WC.
21. Eskandari M. Network slicing in O-RAN-enabled cell-free massive MIMO: A DRL-based power control / M. Eskandari, M. Rahmani Ghourtani, A.G. Burr // Proceedings of the 2025 IEEE Wireless Communications and Networking Conference (WCNC). – Milan: IEEE, 2025. https://doi.org/10.1109/WCNC61545.2025.10978801.
22. Anbazhagan S. Adaptive power-saving mode control in NB-IoT networks using soft actor-critic reinforcement learning for optimal power management / S. Anbazhagan, R.K. Mugelan // Scientific Reports. – 2025. – Vol. 15, Art. 34618. https://doi.org/10.1038/s41598-025-18214-4.
23. Base station power control strategy in ultra-dense networks via deep reinforcement learning / Q. Chen et al // Physical Communication. – 2025. – Vol. 71, Art. 102655. https://doi.org/10.1016/j.phycom.2025.102655.
Review
For citations:
Aibagarov S., Marlambekov D., Mukhanbet A. SAC-BASED UPLINK POWER CONTROL IN 5G NR: DEEP REINFORCEMENT LEARNING FOR JOINT THROUGHPUT AND ENERGY EFFICIENCY OPTIMIZATION. Bulletin of Shakarim University. Technical Sciences. 2026;(3(23)):331-342. https://doi.org/10.53360/2788-7995-2026-3(23)-31
JATS XML















