Authors:
K. N. S. K. Santhosh,Angara Satyam,Kante Satyanarayana,Venkata Raju Athili,Ponugoti Gangadhara Rao,Bhatraju Mahalakshmi Rao,DOI NO:
https://doi.org/10.26782/jmcms.2025.08.00002Keywords:
Deep reinforcement learning,Deep Q-Network,Data Rate,New Radio,Packet Error Rate,Quality of Service,Wireless Networks,Abstract
This paper introduces an intelligent method to enhance communication in unlicensed millimetre-wave (mmWave) networks for New Radio Unlicensed (NR-U) and Wireless Gigabit (WiGig) systems. Since both networks share the same frequency band, they often interfere with each other, reducing performance and fairness. The challenge lies in ensuring smooth coexistence without harming the efficiency of either system. NR-U plays a crucial role in 5G networks by meeting the growing demand for faster wireless communication. To tackle this problem, the authors propose a novel method that integrates two essential processes: codebook selection and user equipment (UE) scheduling. Codebook selection optimizes beam patterns for communication, while UE scheduling determines which users access the network and when. These two processes operate at different speeds, making optimization complex. The researchers use Deep Reinforcement Learning (DRL) to solve this challenge dynamically and intelligently. The proposed system, DeepCBU, is based on a Layered Deep Q-Network (L-DQN) framework. It learns from past experiences to make better decisions over time. DeepCBU adjusts dynamically, balancing the need for high data rates while minimizing interference between NR-U and WiGig. Additionally, it ensures fairness among users by distributing network access efficiently. Simulation results show that DeepCBU outperforms traditional methods like DRL-dirLBT, TS-dirLBT, and TS-DRL. It improves data rates for NR-U, reduces WiGig interference, and better satisfies user Quality of Service (QoS) requirements. Unlike conventional approaches, DeepCBU does not require prior network knowledge, making it highly adaptable. In conclusion, DeepCBU is a powerful DRL-based system that enhances NR-U and WiGig coexistence. It optimizes both codebook selection and UE scheduling, ensuring better performance and fairness in future wireless networks.Refference:
I. Addepalli, T., et al. : ‘Compact MIMO diversity antenna for 5G sub 6 GHz and WLAN (Wi Fi 5 & 6) band applications’. Micromachines. Vol. 14, 2023. 10.1007/s11277-023-10718-4
II. Chinchawade, A. J., et al. : ‘Scheduling in multi hop wireless networks using a distributed learning algorithm’. Proc. 7th Int. Conf. Trends Electron. Informatics (ICOEI)., pp. 1013–1018, 2023. 10.1109/ICOEI56765.2023.10125909
III. Feng, Zheng, Lei Ji, Qiang Zhang, Wei Li. : ‘Spectrum management for mmWave enabled UAV swarm networks: Challenges and opportunities’. IEEE Communications Magazine. Vol. 57, pp. 146–153, 2018. 10.1109/MCOM.2018.1800087
IV. Gao, Xiaoliang, et al. : ‘Challenges in NR U/WiGig coexistence’. IEEE Communications Surveys & Tutorials. Vol. 22, pp. 1234–1256, 2020.
V. Gowtham, R., et al. : ‘Enhancing incentive schemes in edge computing through hierarchical reinforcement learning’. ITEGAM JETIA. Vol. 11, pp. 226–136, 2025. 10.5935/jetia.v11i52.1637
VI. Hu, Honghai, Chen Wang, Yike Gao, Yanan Dong, Qi Chen, Jie Zhang. : ‘On the performance of coexisting NR U and WiGig networks with directional sensing’. IEEE Transactions on Communications. Vol. 73, pp. 469–482, 2025. 10.1109/TCOMM.2024.3430986
VII. Jiang, Li, Zhifeng Li. : ‘Machine learning for NR U spectrum access’. IEEE Internet of Things Journal. Vol. 7, pp. 3220–3231, 2020.
VIII. Kiebel, Sebastian J., Jean Daunizeau, Karl J. Friston. : ‘A hierarchy of time scales and the brain’. PLoS Computational Biology. Vol. 4, pp. e1000209, 2008. 10.1371/journal.pcbi.1000209
IX. Kim, Hongs Up, Minho Lee. : ‘Listen Before Receive for NR U’. IEEE Access. Vol. 9, pp. 23348–23357, 2021.
X. Liu, Xiao, Yilin Chen. : ‘Directional Listen Before Talk for 5G NR U’. IEEE Wireless Communications. Vol. 27, pp. 30–36, 2020.
XI. Mabrouki, Soumaya, Ibraheem Dayoub, Qiang Li, Mourad Berbineau. : ‘Codebook designs for millimeter wave communication systems in both low and high mobility: Achievements and challenges’. IEEE Access. Vol. 10, pp. 25786–25810, 2022.
XII. Milanese, Mauro, Antonella Vicino. : ‘Optimal estimation theory for dynamic systems with set membership uncertainty: An overview’. Automatica. Vol. 27, pp. 997–1006, 1991.
XIII. Mu, Jing, Augusto Di Benedetto. : ‘Networking capability and new product development’. IEEE Transactions on Engineering Management. Vol. 59, pp. 4–19, 2011. 10.1109/TEM.2011.2146256
XIV. Park, Joon Young, et al. : ‘Paired LBT for NR U/WiGig coexistence’. IEEE Transactions on Wireless Communications. Vol. 18, pp. 2448–2462, 2019.
XV. Patriciello, Natale, Sandra Lagen, Biljana Bojović, Lorenza Giupponi. : ‘NR U and IEEE 802.11 technologies coexistence in unlicensed mmWave spectrum: Models and evaluation’. IEEE Access. Vol. 8, pp. 71254–71271, 2020. 10.1109/ACCESS.2020.2987467
XVI. Reddy, M. S., A. Tathababu, S. R. Nallamilli. : ‘Parameters optimization of compact UWB MIMO antenna with WLAN band rejection for short distance wireless communication’. IETE Journal of Research. , 2024. 10.1080/03772063.2025.2483933
XVII. Sharma, R., et al. : ‘Optimization based spectrum access in NR U’. IEEE Transactions on Communications. Vol. 69, pp. 5043–5054, 2021.
XVIII. Srivastava, Amit, Sourav Datta, Sourabh Goyal, Umer Salim, Wajahat J. Hussain, Peng Liu, Shivendra S. Panwar, Rachana Pragada, Persidis Adjakple. : ‘Enhanced distributed resource selection and power control for high frequency NR V2X sidelink’. IEEE Access. Vol. 11, pp. 72756–72780, 2023. 10.1109/ACCESS.2023.3295822
XIX. Ssimbwa, Julius, Byungju Lim, Young Chai Ko. : ‘QoS aware user selection and resource assignment for coexistence of NR U and Wi Fi enabled IoT networks’. IEEE Internet of Things Journal. Vol. 11, pp. 30293–30308, 2024. 10.1109/JIOT.2024.3410687
XX. Sun, Jie, et al. : ‘DRL based access protocols for coexistence’. IEEE Journal on Selected Areas in Communications. Vol. 39, pp. 1124–1138, 2021.
XXI. Wang, Yi, Jian Li, Lei Huang, Yao Jing, Andreas Georgakopoulos, Panagiotis Demestichas. : ‘5G Mobile: spectrum broadening to higher frequency bands to support high data rates’. IEEE Vehicular Technology Magazine. Vol. 9, pp. 39–46, 2014. 10.1109/MVT.2014.2333694
XXII. Xu, Bin, et al. : ‘Online learning for codebook optimization’. IEEE Transactions on Signal Processing. Vol. 70, pp. 1422–1435, 2022.
XXIII. Ye, Xuan, Li Fu. : ‘Joint codebook selection and UE scheduling for unlicensed mmWave NR U/WiGig coexistence based on deep reinforcement learning’. IEEE Transactions on Mobile Computing. pp.1-16, 2024. 10.1109/TMC.2024.3356442
XXIV. Zhang, Meng, Rajkumar Ranjan, Markus Menzel, Surya Nepal, Paul Strazdins, Wai Jie, Lingfeng Wang. : ‘An infrastructure service recommendation system for cloud applications with real time QoS requirement constraints’. IEEE Systems Journal. Vol. 11, pp. 2960–2970, 2015.
XXV. Zhang, Yuan, et al. : ‘Deep learning for LTE LAA/Wi Fi coexistence’. IEEE Transactions on Vehicular Technology. Vol. 68, pp. 2345–2358, 2019.
XXVI. 3GPP. : ‘NR U Study Item Technical Report’. 3GPP TR 38.889. Vol. 2019.