Service offloading method with deep reinforcement learning in edge computing empowered Internet of vehicles

doi:10.13196/j.cims.2022.10.026

Computer Integrated Manufacturing System ›› 2022, Vol. 28 ›› Issue (10): 3304-3315.DOI: 10.13196/j.cims.2022.10.026

Previous Articles Next Articles

Service offloading method with deep reinforcement learning in edge computing empowered Internet of vehicles

LIU Guozhi¹,DAI Fei¹⁺,MO Qi²,XU Xiaolong³,QIANG Zhenping¹,WANG Leiguang¹

1.School of Big Data and Intelligent Engineering,Southwest Forestry University
2.School of Software,Yunnan University
3.School of Computer and Software,Nanjing University of Information Science and Technology

Online:2022-10-31 Published:2022-11-10
Supported by:
Project supported by the National Natural Science Foundation,China(No.62262063,12163004),the Key Science Foundation for Basic Research of Yunnan Province,China(No.202101AS070007),the Dou Wanchun Expert Workstation of Yunnan Province,China(No.202205AF150013),the Science and Technology Youth Lift Talents of Yunnan Province,China(No.61862065),the Major Project of Science and Technology of Yunnan Province,China(No.202002AD080002),and the Science and Technology Department of Yunnan Province,China(No.202001AT070135).

车辆边缘计算环境下基于深度强化学习的服务卸载方法

刘国志¹,代飞¹⁺,莫启²,许小龙³,强振平¹,王雷光¹

1.西南林业大学大数据与智能工程学院
2.云南大学软件学院
3.南京信息工程大学计算机与软件学院

基金资助:
国家自然科学基金资助项目(62262063,12163004);云南省基础研究重点资助项目(202101AS070007);云南省窦万春专家工作站资助项目(202205AF150013);云南省科学技术协会青年科技人才资助项目(61862065);云南省科技重大资助项目(202002AD080002);云南省科技厅资助项目(202001AT070135)。

Abstract

Abstract: Vehicular edge computing is a new computing paradigm.To make the service offloading efficiently under vehicular edge environment,by both considering the service offloading strategy and the collaborative allocation of edge server and cloud server,a Deep Q-network (DQN) based Service Offloading DQN (SODQN)algorithm was proposed.An End-Edge-Cloud architecture was proposed for service offloading and the problem of service offloading was formulated as the optimization problem under the constraints of the computing and communication resources of the edge server.DQN was used to solve the optimization problem,where greedy algorithm,experience replay mechanism and double DQN mechanism were introduced in the learning process.The extensive simulation experiments were conducted,and the experimental results showed that the proposed offloading scheme could achieve a good performance.

Key words: service offloading, end-edge-cloud architecture, deep Q-network, deep reinforcement learn, edge computing

摘要： 为了在车辆边缘环境下高效地进行服务卸载,同时考虑服务的卸载决策以及边缘服务器和云服务器的协同资源分配,提出一种基于深度强化学习的服务卸载方法。首先提出车辆边缘环境下一种端—边—云协同的服务卸载架构,将服务卸载问题归约为边缘服务器计算和通信资源约束下获得最小平均服务时延的优化问题;然后引入深度Q网络解决优化问题,在学习过程中引入贪婪算法、经验回放机制和双网络机制。通过实验表明,所提方法具有可行性,所提卸载方案性能良好。

关键词: 服务卸载, 端—边—云架构, 深度Q网络, 深度强化学习, 边缘计算

CLC Number:

TP311

LIU Guozhi, DAI Fei, MO Qi, XU Xiaolong, QIANG Zhenping, WANG Leiguang. Service offloading method with deep reinforcement learning in edge computing empowered Internet of vehicles[J]. Computer Integrated Manufacturing System, 2022, 28(10): 3304-3315.

刘国志, 代飞, 莫启, 许小龙, 强振平, 王雷光. 车辆边缘计算环境下基于深度强化学习的服务卸载方法[J]. 计算机集成制造系统, 2022, 28(10): 3304-3315.

[1]	YANG Qisen, WANG Shenzhi, SANG Jinnan, WANG Chaofei, HUANG Gao, WU Cheng, SONG Shiji. Path planning and real-time obstacle avoidance methods of intelligent ships in complex open water environment [J]. Computer Integrated Manufacturing System, 2022, 28(7): 2030-2040.
[2]	YAN Hanzhi, XU Xiaolong, DAI Fei, QI Lianyong, DOU Wanchun, LI Tong. Edge server deployment strategy with reinforcement learning in Internet of vehicles [J]. Computer Integrated Manufacturing System, 2022, 28(10): 3146-3155.
[3]	CHEN Qiaoxin, LU Yu, LIN Bing, WANG Suyun, SHAO Xun. Real-time scheduling strategy for reasoning tasks in vehicle edge computing [J]. Computer Integrated Manufacturing System, 2022, 28(10): 3295-3303.
[4]	. Hierarchical filtering algorithm for distributed abnomaly data based on urban computing [J]. , 2021, 27(9): 2525-2531.
[5]	. Parallel task scheduling algorithm based on collaborative device and edge in UAV delivery system [J]. , 2021, 27(9): 2575-2582.
[6]	. Adaptive mobile path-aware user allocation algorithm under edge computing environment [J]. , 2021, 27(9): 2592-2603.
[7]	. Federated learning based method for intelligent computing with privacy preserving in edge computing [J]. , 2021, 27(9): 2604-2610.
[8]	. Deep reinforcement learning method for biped robot gait control [J]. , 2021, 27(8): 2341-2349.
[9]	. Fault-tolerant scheduling algorithm for service workflow in MEC environment [J]. , 2021, 27(6): 1693-1702.
[10]	. Visualization management and control methods of production cell based on digital twin [J]. , 2021, 27(5): 1256-1268.
[11]	. New industrial data processing architecture and multi-scenario applications for aviation digital workshop [J]. , 2021, 27(4): 1021-1031.
[12]	. Predictive maintenance decision-making for serial production lines based on deep reinforcement learning [J]. , 2021, 27(12): 3416-3428.
[13]	. Perception and access of manufacturing resources and intelligent gateway technology for edge computing [J]. , 2020, 26(第1): 40-48.
[14]	. Security and energy aware scheduling for service workflow in mobile edge computing [J]. , 2020, 26(7): 1831-1842.
[15]	. Energy efficient computing task offloading strategy for deep neural networks in mobile edge computing [J]. , 2020, 26(6): 1607-1615.

Service offloading method with deep reinforcement learning in edge computing empowered Internet of vehicles

车辆边缘计算环境下基于深度强化学习的服务卸载方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics