留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于流量模式的Q-学习路由及其连接调度

姚铭明 曹霑懋 黄启嵩 单志龙

姚铭明, 曹霑懋, 黄启嵩, 单志龙. 基于流量模式的Q-学习路由及其连接调度[J]. 华南师范大学学报(自然科学版), 2021, 53(4): 107-114. doi: 10.6054/j.jscnun.2021065
引用本文: 姚铭明, 曹霑懋, 黄启嵩, 单志龙. 基于流量模式的Q-学习路由及其连接调度[J]. 华南师范大学学报(自然科学版), 2021, 53(4): 107-114. doi: 10.6054/j.jscnun.2021065
YAO Mingming, CAO Zhanmao, HUANG Qisong, SHAN Zhilong. Q-Learning Routing and Link Scheduling Based on Traffic Mode[J]. Journal of South China normal University (Natural Science Edition), 2021, 53(4): 107-114. doi: 10.6054/j.jscnun.2021065
Citation: YAO Mingming, CAO Zhanmao, HUANG Qisong, SHAN Zhilong. Q-Learning Routing and Link Scheduling Based on Traffic Mode[J]. Journal of South China normal University (Natural Science Edition), 2021, 53(4): 107-114. doi: 10.6054/j.jscnun.2021065

基于流量模式的Q-学习路由及其连接调度

doi: 10.6054/j.jscnun.2021065
基金项目: 

国家自然科学基金项目 61671213

广州市科技计划项目 202007040006

详细信息
    通讯作者:

    曹霑懋, Email: caozhanmao@m.scnu.deu.cn

  • 中图分类号: TP393

Q-Learning Routing and Link Scheduling Based on Traffic Mode

  • 摘要: 为解决无线网状网中因多条路径同时传输数据而引起网络性能降低的问题, 提出了一个基于流量的Q-学习路由与调度方案(QRST): 针对每一个路由请求, 首先采用强化学习中的Q-学习算法寻找路径; 然后根据找到的路径结合信道分配完成组合调度, 以启发式的方法尽可能为每个时隙使用网络资源分配路径的连接. 并在不同网络资源配置和多种流量请求下进行虚拟计算实验, 以验证该方案的正确性和有效性. 实验结果表明: 与COSS方案和AODV方案相比,采用QRST方案的无线网状网在吞吐量、激活链路数量和传输完成时间等网络性能上有较好的表现.
  • 图  1  强化学习模型图解

    Figure  1.  The illustration of the reinforcement learning model

    图  2  QRST方案的路由模型

    Figure  2.  The routing model of QRST scheme

    图  3  36个节点的随机网络拓扑图

    Figure  3.  The 36-node random topology

    图  4  采用3种方案的无线网状网在不同数量路由请求下的平均吞吐量(|R|=16∧|C|=32)

    Figure  4.  The average throughput of WMN using three methods under different numbers of routing request (|R|=16∧|C|=32)

    图  5  采用3种方案的无线网状网在不同数量路由请求下的平均激活链路数量(|R|=16∧|C|=32)

    Figure  5.  The average number of active links of WMN using three methods under different numbers of routing request (|R|=16∧|C|=32)

    图  6  采用3种方案的无线网状网在不同数量路由请求下的传输时间(|R|=16∧|C|=32)

    Figure  6.  The transmit time of WMN using three methods under different numbers of routing request (|R|=16∧|C|=32)

    图  7  采用QRST方案的无线网状网在不同迭代次数下的多组路由请求的平均吞吐量(|R|=16∧|C|=32)

    Figure  7.  The average throughput of different routing requests under different iteration times of WMN using QRST (|R|=16∧|C|=32)

    图  8  30对路由请求的学习过程对平均吞吐量的影响(|R|=4∧|C|=8)

    Figure  8.  The influence of learning process of 30 pairs of routing requests on average throughout (|R|=4∧|C|=8)

    图  9  |C|=8且不同接口数时采用3种方案的无线网状网的平均吞吐量

    Figure  9.  The average throughput of WMN using three methods when |C|=8 and under different interface numbers

    图  10  |C|=16且不同接口数时采用3种方案的无线网状网的平均吞吐量

    Figure  10.  The average throughput of WMN using three methods when |C|=16 and under different interface numbers

    图  11  |C|=32且不同接口数时采用3种方案的无线网状网的平均吞吐量

    Figure  11.  The average throughput of WMN using three methods when |C|=32 and under different interface numbers

  • [1] AKYILDIZ I F, WANG X D, WANG W L. Wireless mesh networks: a survey[J]. Computer Networks, 2004, 47(4): 445-487.
    [2] GALVEEZ J J, RUIZ P M. Efficient rate allocation, routing and channel assignment in wireless mesh networks supporting dynamic traffic flows[J]. Ad Hoc Networks, 2013, 11(6): 1765-1781. doi: 10.1016/j.adhoc.2013.04.002
    [3] CAO Z M, WU C Q, BERRY M L. On routing of multiple concurrent user requests in multi-radio multi-channel wireless mesh networks[C]//Proceedings of the IEEE 17th International Conference on Parallel and Distributed Computing, Applications and Technologies. Piscataway: IEEE, 2016: 24-29.
    [4] GHASEMPOUR A. Internet of things in smart grid: architecture, applications, services, key technologies, and cha-llenges[J]. Inventions, 2019, 4(1): 22-34. doi: 10.3390/inventions4010022
    [5] ROH T, LEE J W. Channel assignment, link scheduling, routing, and rate control for multi-channel wireless mesh networks with directional antennas[J]. Journal of Communications and Networks, 2016, 18(6): 884-891. doi: 10.1109/JCN.2016.000123
    [6] JONATHAN W, XUE Y. The robust joint solution for channel assignment and routing for wireless mesh networks with time partitioning[J]. Ad Hoc Networks, 2014, 13: 210-221. doi: 10.1016/j.adhoc.2011.05.002
    [7] JACQUET P, MUHLETHALER P, CLAUSEN T, et al. Optimized link state routing protocol for ad hoc networks[C]//Proceedings of the 2001 IEEE International Multi Topic Conference on Technology for the 21st Century. Lahore, Pakistan: IEEE, 2001: 62-68.
    [8] PENG Y H, YAO Y, GUO L, et al. An efficient joint channel assignment and QoS routing protocol for IEEE 802.11 multi-radio multi-channel wireless mesh networks[J]. Journal of Network and Computer Applications, 2013, 36(2): 843-857. doi: 10.1016/j.jnca.2012.11.003
    [9] ISLAM M, RAZZAQUE M A, MAMUN-OR-RASHID M. Joint link-channel selection and power allocation in multi-radio wireless mesh networks[C]//Proceedings of the 2017 IEEE 18th International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM). Macau, China: IEEE, 2017: 1-7.
    [10] SUTTON R S, BARTO A G. Reinforcement learning: an introduction[M]. Cambridge: MIT Press, 2018.
    [11] WU C, YOSHINAGA T, JI Y, et al. A reinforcement lear-ning-based data storage scheme for vehicular ad hoc networks[J]. IEEE Transactions on Vehicular Technology, 2016, 66(7): 6336-6348.
    [12] SAVAGLIO C, PACE P, ALOI G, et al. Lightweight reinforcement learning for energy efficient communications in wireless sensor networks[J]. IEEE Access, 2019, 7: 29355-29364. doi: 10.1109/ACCESS.2019.2902371
    [13] CHEN X, ZHAO Z, ZHANG H, et al. Reinforcement lear-ning enhanced iterative power allocation in stochastic cognitive wireless mesh networks[J]. Wireless Personal Communications, 2011, 57(1): 89-104. doi: 10.1007/s11277-010-0008-6
    [14] BOUSHABA M, HAFID A, BELBEKKOUCHE A, et al. Reinforcement learning based routing in wireless mesh networks[J]. Wireless Networks, 2013, 19(8): 2079-2091. doi: 10.1007/s11276-013-0592-y
    [15] COUTINHO N, MATOS R, MARQUES C, et al. Dynamic dual-reinforcement-learning routing strategies for quality of experience-aware wireless mesh networking[J]. Computer Networks, 2015, 88: 269-285. doi: 10.1016/j.comnet.2015.06.016
    [16] 黄启嵩, 曹霑懋, 单志龙. 无线网状网中多并发流路由结合调度的组合方案[J]. 华南师范大学学报(自然科学版), 2020, 52(1): 122-128. doi: 10.6054/j.jscnun.2020018

    HUANG Q S, CAO Z M, SHAN Z L. A combined scheme of multi concurrent flow routing and scheduling in wireless mesh networks[J]. Journal of South China Normal University(Natural Science Edition), 2020, 52(1): 122-128. doi: 10.6054/j.jscnun.2020018
    [17] WATKINS C J C H, DAYAN P. Q-learning[J]. Machine Learning, 1992, 8(3/4): 279-292. doi: 10.1023/A:1022676722315
    [18] CAO Z M, XIAO W J, PENG L. A mesh×chain graph model for MIMO scheduling in IEEE802.16 WMN[C]//Proceedings of the 2010 Second International Conference on Computer Modeling and Simulation. Sanya, Hainan, China: IEEE, 2010: 547-551.
    [19] CAO Z M, WU C Q, ZHANG Y P, et al. On modeling and analysis of mimo wireless mesh networks with triangular overlay topology[J]. Mathematical Problems in Enginee-ring, 2015, 2015: 185262/1-11.
  • 加载中
图(11)
计量
  • 文章访问数:  78
  • HTML全文浏览量:  15
  • PDF下载量:  11
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-01-06
  • 网络出版日期:  2021-09-03
  • 刊出日期:  2021-08-25

目录

    /

    返回文章
    返回