Abstract: To solve the problems of delay fluctuation and increased energy consumption caused by offloading computational tasks to the remote cloud in the Internet of vehicles scenario,an Internet of vehicles system model integrating software defined networking and mobile edge computing was built.To minimize the delay and energy consumption of computation offloading in the Internet of vehicles,a dynamic computation offloading strategy based on deep reinforcement learning was designed.The strategy was modeled in two dimensions of energy consumption and delay of computational tasks,and an improved algorithm of adding Softmax and prioritized experience replay based on the traditional twin delayed deep deterministic policy gradient was proposed,which named Twin Delayed Deep Deterministic policy gradient with Softmax and Prioritized experience replay (SP-TD3).The simulation experimental results showed that the proposed algorithm had superior performance for energy consumption and delay under different vehicle numbers.

Key words: edge computing, deep reinforcement learning, computation offloading, software defined networking

摘要： 针对车联网场景下计算任务卸载至远端云中造成的延迟波动和能耗增加问题,搭建了融合软件定义网络和移动边缘计算的车联网系统模型。为了最小化车联网中计算卸载的时延和能耗,设计了一种基于深度强化学习的动态计算卸载策略。该策略从计算任务的能耗和时延这两个维度进行建模,并提出在传统的双延迟深度确定性策略梯度的基础上加入Softmax和优先经验回放的改进算法(SP-TD3)。仿真实验结果表明,设计的计算卸载策略在不同车辆数下对于能耗和时延有较为优越的性能。

关键词: 边缘计算, 深度强化学习, 计算卸载, 软件定义网络