Q-Learning Routing and Link Scheduling Based on Traffic Mode

YAO Mingming; CAO Zhanmao; HUANG Qisong; SHAN Zhilong

doi:10.6054/j.jscnun.2021065

Journal of South China Normal University (Natural Science Edition) > 2021 > 53(4): 107-114. > DOI: 10.6054/j.jscnun.2021065

YAO Mingming, CAO Zhanmao, HUANG Qisong, SHAN Zhilong. Q-Learning Routing and Link Scheduling Based on Traffic Mode[J]. Journal of South China Normal University (Natural Science Edition), 2021, 53(4): 107-114. DOI: 10.6054/j.jscnun.2021065

Citation:

PDF (1243 KB)

Q-Learning Routing and Link Scheduling Based on Traffic Mode

School of Computer, South China Normal University, Guangzhou 510631, China

More Information

Received Date: January 05, 2021
Available Online: September 02, 2021

Graphical Abstract

Abstract

Abstract

The interference and resource congestion caused by multiple concurrent flows may cause sharp perfor-mance degradation of wireless mesh networks. In order to solve the problem, a Q-learning Routing and Scheduling concerning Traffic (QRST) scheme is proposed. Firstly, the Q-learning algorithm is used to find the path for each routing request. Then the combined scheduling is completed according to the path finding and channel allocation, and the connection of paths is allocated with cyber source for every slot in a heuristic way. In order to verify the co-rrectness and effectiveness of the scheme, virtual computing is performed under different network resource configurations and multiple traffic requests. The experimental results show that, compared with COSS and AODV, wireless mesh network using the QRST scheme has better performance in terms of throughput, activated link number and transmission completion time.
- wireless mesh network,
- routing,
- reinforcement learning,
- Q-learning,
- multiple concurrent flows

FullText(HTML)

References (19)

References

[1]	AKYILDIZ I F, WANG X D, WANG W L. Wireless mesh networks: a survey[J]. Computer Networks, 2004, 47(4): 445-487.
[2]	GALVEEZ J J, RUIZ P M. Efficient rate allocation, routing and channel assignment in wireless mesh networks supporting dynamic traffic flows[J]. Ad Hoc Networks, 2013, 11(6): 1765-1781. doi: 10.1016/j.adhoc.2013.04.002
[3]	CAO Z M, WU C Q, BERRY M L. On routing of multiple concurrent user requests in multi-radio multi-channel wireless mesh networks[C]//Proceedings of the IEEE 17th International Conference on Parallel and Distributed Computing, Applications and Technologies. Piscataway: IEEE, 2016: 24-29.
[4]	GHASEMPOUR A. Internet of things in smart grid: architecture, applications, services, key technologies, and cha-llenges[J]. Inventions, 2019, 4(1): 22-34. doi: 10.3390/inventions4010022
[5]	ROH T, LEE J W. Channel assignment, link scheduling, routing, and rate control for multi-channel wireless mesh networks with directional antennas[J]. Journal of Communications and Networks, 2016, 18(6): 884-891. doi: 10.1109/JCN.2016.000123
[6]	JONATHAN W, XUE Y. The robust joint solution for channel assignment and routing for wireless mesh networks with time partitioning[J]. Ad Hoc Networks, 2014, 13: 210-221. doi: 10.1016/j.adhoc.2011.05.002
[7]	JACQUET P, MUHLETHALER P, CLAUSEN T, et al. Optimized link state routing protocol for ad hoc networks[C]//Proceedings of the 2001 IEEE International Multi Topic Conference on Technology for the 21st Century. Lahore, Pakistan: IEEE, 2001: 62-68.
[8]	PENG Y H, YAO Y, GUO L, et al. An efficient joint channel assignment and QoS routing protocol for IEEE 802.11 multi-radio multi-channel wireless mesh networks[J]. Journal of Network and Computer Applications, 2013, 36(2): 843-857. doi: 10.1016/j.jnca.2012.11.003
[9]	ISLAM M, RAZZAQUE M A, MAMUN-OR-RASHID M. Joint link-channel selection and power allocation in multi-radio wireless mesh networks[C]//Proceedings of the 2017 IEEE 18th International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM). Macau, China: IEEE, 2017: 1-7.
[10]	SUTTON R S, BARTO A G. Reinforcement learning: an introduction[M]. Cambridge: MIT Press, 2018.
[11]	WU C, YOSHINAGA T, JI Y, et al. A reinforcement lear-ning-based data storage scheme for vehicular ad hoc networks[J]. IEEE Transactions on Vehicular Technology, 2016, 66(7): 6336-6348.
[12]	SAVAGLIO C, PACE P, ALOI G, et al. Lightweight reinforcement learning for energy efficient communications in wireless sensor networks[J]. IEEE Access, 2019, 7: 29355-29364. doi: 10.1109/ACCESS.2019.2902371
[13]	CHEN X, ZHAO Z, ZHANG H, et al. Reinforcement lear-ning enhanced iterative power allocation in stochastic cognitive wireless mesh networks[J]. Wireless Personal Communications, 2011, 57(1): 89-104. doi: 10.1007/s11277-010-0008-6
[14]	BOUSHABA M, HAFID A, BELBEKKOUCHE A, et al. Reinforcement learning based routing in wireless mesh networks[J]. Wireless Networks, 2013, 19(8): 2079-2091. doi: 10.1007/s11276-013-0592-y
[15]	COUTINHO N, MATOS R, MARQUES C, et al. Dynamic dual-reinforcement-learning routing strategies for quality of experience-aware wireless mesh networking[J]. Computer Networks, 2015, 88: 269-285. doi: 10.1016/j.comnet.2015.06.016
[16]	黄启嵩, 曹霑懋, 单志龙. 无线网状网中多并发流路由结合调度的组合方案[J]. 华南师范大学学报(自然科学版), 2020, 52(1): 122-128. doi: 10.6054/j.jscnun.2020018 HUANG Q S, CAO Z M, SHAN Z L. A combined scheme of multi concurrent flow routing and scheduling in wireless mesh networks[J]. Journal of South China Normal University(Natural Science Edition), 2020, 52(1): 122-128. doi: 10.6054/j.jscnun.2020018
[17]	WATKINS C J C H, DAYAN P. Q-learning[J]. Machine Learning, 1992, 8(3/4): 279-292. doi: 10.1023/A:1022676722315
[18]	CAO Z M, XIAO W J, PENG L. A mesh×chain graph model for MIMO scheduling in IEEE802.16 WMN[C]//Proceedings of the 2010 Second International Conference on Computer Modeling and Simulation. Sanya, Hainan, China: IEEE, 2010: 547-551.
[19]	CAO Z M, WU C Q, ZHANG Y P, et al. On modeling and analysis of mimo wireless mesh networks with triangular overlay topology[J]. Mathematical Problems in Enginee-ring, 2015, 2015: 185262/1-11.

Cited By

Get Citation

PDF

XML

Article views (498) PDF downloads (72)

Turn off MathJax

Article Contents

Abstract

References

Q-Learning Routing and Link Scheduling Based on Traffic Mode

Abstract

References

Catalog

Export File

Citation

Format

Content