Super Frame Time Slot Allocation Algorithm for Dual-Mode Communication Based on Deep-Reinforcement Learning

Junhao FENG; Xuezi ZHAN; Yufan YAN; Pengjiao WANG; Zhixiong CHEN

doi:10.13648/j.cnki.issn1674-0629.2026.07.006

PDF(1906 KB)

South Power Sys Technol ›› 2026, Vol. 20 ›› Issue (7) : 58-66. DOI: 10.13648/j.cnki.issn1674-0629.2026.07.006

System Analysis & Operation

Super Frame Time Slot Allocation Algorithm for Dual-Mode Communication Based on Deep-Reinforcement Learning

Junhao FENG ¹^,² ,
Xuezi ZHAN ³ ,
Yufan YAN ⁴ ,
Pengjiao WANG ⁴ ,
Zhixiong CHEN ³^,⁴

Author information +

History +

Abstract

The hybrid networking of power line and wireless dual-mode communication can achieve complementary advantages, demonstrating significant application prospects in fields such as the Internet of Things for power systems and smart homes. To enhance the access flexibility and network resource utilization of dual-mode communication MAC algorithms, a superframe time slot resource allocation algorithm for dual-mode gateways based on deep reinforcement learning is proposed. Firstly, an interaction model between the dual-mode communication gateway and terminals is established, detailing the superframe structure and the specific execution steps of each phase. Secondly, key reward functions, state spaces, and action spaces for machine learning applications are defined, with the gateway monitoring and collecting parameters such as superframe access throughput. Through iterative learning and training, optimized parameters including the number and proportion of superframe time slots are obtained. Finally, simulations are conducted to verify the effectiveness and reliability of the proposed algorithm. Comparisons are made with fixed superframe structure and non-superframe structure algorithms, analyzing network throughput, average delay, packet loss rate under different algorithms, as well as the influence patterns of key parameters on system performance. Simulation results indicate that the proposed algorithm can effectively improve system performance in terms of throughput and delay, enabling flexible and efficient resource allocation.

Key words

dual-mode communication / double deep Q-network / MAC layer access / super frame structure / time slot allocation

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

Junhao FENG , Xuezi ZHAN , Yufan YAN , et al . Super Frame Time Slot Allocation Algorithm for Dual-Mode Communication Based on Deep-Reinforcement Learning[J]. Southern Power System Technology. 2026, 20(7): 58-66 https://doi.org/10.13648/j.cnki.issn1674-0629.2026.07.006

References

List( Publishing order | Descend order by publishing year | Descend order by cited within ) Chart analysis

[1]	DIB DE M B A L， FERNANDES V， DE L. FILOMENO M，et al. Hybrid PLC/wireless communication for smart grids and internet of things applications［J］. IEEE Internet of Things Journal， 2018， 5（2）： 655 - 667. Cited in this article [1]

[2]	代洲，张全玲，凌琴，等.面向电力线通信融合双模态索引和时间分集的差分混沌移位方案［J］.电力系统保护与控制，2025，53（15）：135 - 145. DAI Zhou， ZHANG Quanling， LING Qin，et al.Dual mode index and time diversity based DCSK for power line communications［J］. Power System Protection and Control，2025，53（15）：135 - 145. Cited in this article [1]

[3]

王清，荆臻，李琮琮，等. 宽带电力线载波通信物理层资源分配新算法［J］. 电测与仪表， 2021， 58（7）： 124 - 129.

WANG

Qing

， JING

Zhen

， LI

Congcong

， et al. A novel algorithm of physical layer resource allocation for broadband power line carrier communication［J］. Electrical Measurement & Instrumentation， 2021， 58（7）： 124 - 129.

Cited in this article [1]

[4]	YOON S G， JANG S， KIM Y H， et al. Opportunistic routing for smart grid with power line communication access networks［J］. IEEE Transactions on Smart Grid， 2014， 5（1）： 303 - 311.

[5]	PAPADOPOULOS T A， KALOUDAS C G， CHRISTOS A I， et al. Application of narrowband power-line communication in medium-voltage smart distribution grids［J］. IEEE Transactions on Power Delivery， 2013， 28（2）： 981 - 988. Cited in this article [1]

[6]	杨璐，吴清亮. 无线接入网络自适应公平调度算法［J］. 通信学报， 2012， 33（1）： 102 - 106，113. YANG Lu， WU Qingliang. Self-adaptive fair scheduling algorithm in wireless network［J］. Journal on Communications， 2012， 33（1）： 102 - 106，113. Cited in this article [1]

[7]	陈生海，言小琴，黎赛，等. 混合双跳PLC-FSO通信系统的性能分析［J］. 通信学报， 2021， 42（10）： 243 - 250. CHEN Shenghai， YAN Xiaoqin， LI Sai， et al. Performance analysis of dual-hop PLC-FSO communication system［J］， Journal on Communication， 2021， 42（10）： 243 - 250. Cited in this article [1]

[8]	曹旺斌，唐宏凯，谢志远，等. 下行NOMA-PLC系统最优功率分配方法研究［J］. 中国电机工程学报， 2025， 45（3）： 834 - 846. CAO Wangbin， TANG Hongkai， XIE Zhiyuan， et al. Research on optimal power allocation scheme in downlink NOMA-PLC system［J］. Proceedings of the CSEE， 2025， 45（3）： 834 - 846. Cited in this article [1]

[9]

黄鑫，卞宝银，杨鸿珍，等. 无线和电力线混合通信的MAC层接入算法设计和性能分析［J］. 电力信息与通信技术， 2024， 22（1）： 77 - 85.

HUANG

Xin

， BIAN

Baoyin

， YANG

Hongzhen

， et al. Design and performance analysis of MAC layer access algorithms for hybrid communication of wireless and power line［J］. Electric Power Information and Communication Technology， 2024， 22（1）： 77 - 85.

Cited in this article [2]

[10]	CHEN Z， ZHI L， ZHAO X， et al. Parallel access scheme for wireless and power line dual-connection communication［J］. IEEE Transactions on Vehicular Technology， 2022， 71（7）： 7997 - 8001. Cited in this article [2]

[11]	AL-JANABI T A， AL-RAWESHIDY H S. An energy efficient hybrid MAC protocol with dynamic sleep-based scheduling for high density IoT networks［J］. IEEE Internet of Things Journal， 2019， 6（2）： 2273 - 2287. Cited in this article [4]

[12]	SHRESTHA B， HOSSAIN E， CHOI K W. Distributed and centralized hybrid CSMA/CA-TDMA schemes for single-hop wireless networks［J］. IEEE Transactions on Wireless Communications， 2014， 13（7）： 4050 - 4065. Cited in this article [3]

[13]	MNIH V， KAVUKCUOGLU K， SILVER D， et al. Human-level control through deep reinforcement learning［J］. Nature， 2015， 518（7540）： 529 - 533. Cited in this article [2]

[14]	NGUYEN T T， NGUYEN N D， NAHAVANDI S. Deep reinforcement learning for multiagent systems： a review of challenges， solutions， and applications［J］. IEEE Transactions on Cybernetics， 2020， 50（9）： 3826 - 3839. Cited in this article [1]

[15]	赵晓荣. 基于深度强化学习的异构网络资源分配［D］. 济南：山东大学， 2023. Cited in this article [1]

[16]

姜华，杨家伟，黄巍，等. 基于深度强化学习的D2D辅助MEC网络资源分配算法［J］. 电力信息与通信技术， 2023， 21（7）： 51 - 58.

JIANG

Hua

， YANG

Jiawei

， HUANG

Wei

， et al. A D2D-assisted MEC network resource allocation algorithm based on deep reinforcement learning［J］. Electric Power Information and Communication Technology， 2023， 51（7）： 51 - 58.

Cited in this article [1]

[17]	刘乔寿，周雄，刘爽，等. 基于深度强化学习的OFDM自适应导频设计［J］. 通信学报， 2023， 44（9）： 104 - 114. LIU Qiaoshou， ZHOU Xiong， LIU Shuang， et al. Adaptive pilot design for OFDM based on deep reinforcement learning［J］. Journal on Communications， 2023， 44（9）： 104 - 114. Cited in this article [1]

[18]

赵玉新，杜登辉，成小会，等. 基于强化学习的海洋移动观测网络观测路径规划方法［J］. 智能系统学报， 2022， 17（1）： 192 - 200.

ZHAO

Yuxin

， DU

Denghui

， CHENG

Xiaohui

， et al. Path planning for mobile ocean observation network based on reinforcement learning［J］. CAAI Transactions on Intelligent Systems， 2022， 17（1）： 192 - 200.

Cited in this article [1]

[19]

何云，申敏，王蕊，等. 无蜂窝毫米波大规模MIMO系统基于深度强化学习的节能睡眠策略［J］. 电子学报， 2023， 51（10）： 2831 - 2843.

Yun

， SHEN

Min

， WANG

Rui

， et al. Energy-efficient sleep-mode based on deep reinforcement learning for cell-free mmWave massive MIMO systems［J］. Acta Electronica Sinica， 2023， 51（10）： 2831 - 2843.

Cited in this article [1]

[20]	PEI Z， CHEN W， DU L. MAC Contention protocol based on reinforcement learning for IoV communication environments［C］//2021 IEEE 6th International Conference on Computer and Communication Systems （ICCCS）， April 23 - 26， 2021， Chengdu， China. New York： IEEE， 2021： 546 - 552. Cited in this article [1]

[21]	YU Y， LIEW S C， WANG T. Non-uniform time-step deep q-network for carrier-sense multiple access in heterogeneous wireless network［J］. IEEE Transactions on Mobile Computing， 2021， 20（9）： 2848 - 2861. Cited in this article [1]

[22]	YE X， YU Y， FU L. Deep reinforcement learning based MAC protocol for underwater acoustic networks［J］. IEEE Transactions on Mobile Computing， 2022， 21（5）： 1625 - 1638. Cited in this article [1]

[23]	贺文晨，郭少勇，邱雪松，等. 基于DRL的联邦学习节点选择方法［J］. 通信学报， 2021， 42（6）： 62 - 71. HE Wenchen， GUO Shaoyong， QIU Xuesong， et al. Node selection method in federated learning based on deep reinforcement learning［J］. Journal on Communications， 2021， 42（6）： 62 - 71. Cited in this article [3]

[24]	VLACHOU C， BANCHS A， SALVADOR P， et al. Analysis and enhancement of CSMA/CA with deferral in power-line communications［J］. IEEE Journal on Selected Areas in Communications， 2016， 34（7）： 1978 - 1991. Cited in this article [2]

[25]	CHEN Z， ZHI L， CHEN P， et al. An MAC layer algorithm based on power line-wireless dual media channels and multiplexing［J］. China Communications， 2022， 19（10）： 78 - 91. Cited in this article [1]

[26]	刘颖，杨鹏飞，张立军，等. 前馈神经网络和循环神经网络的鲁棒性验证综述［J］. 软件学报， 2023， 34（7）： 3134 - 3166. LIU Yin， YANG Pengfei， ZHANG Lijun， et al. Survey on robustness verification of feedforward neural networks and recurrent neural networks［J］. Journal of Software， 2023， 34（7）： 3134 - 3166. Cited in this article [1]