关键词: 双足机器人, 步态控制, 深度强化学习, 智能体, 操作—评论, 改进深度Q网络算法

Abstract: Aiming at the stable control of gait during biped robot walking,a deep reinforcement learning method with improved Deep Q-Network (DQN) was proposed.By combining DQN algorithm with a deterministic strategy gradient,an improved DQN learning network was proposed to replace the critic network of actor-critic network with a clipped Double-Q network.A link model of biped robot was established,and the proposed network was used for biped robots gait control training as agents in a conventional flat road environment.MATLAB simulation results showed that compared with DQN and Deep Deterministic Policy Gradient (DDPG) algorithms,the proposed algorithm had a better training speed and its average reward curve had a good smoothness.Under the CPU training conditions,the agent training could be completed after about 20 hours of deep reinforcement learning.The biped robot could achieve stable and fast walking (average speed about 0.5m/s) under the conditions of small torque and long distance (about 5 meters).

Key words: biped robot, gait control, deep reinforcement learning, agent, actor-critic, improved deep Q-net algorithm