Abstract: To address the problem of decreased grasping success rates in complex scenarios where there are obvious obstructions and overlapping objects,a multi-view grasping and pushing collaborative strategy based on Deep Reinforcement Learning was proposed.This strategy utilized 3D point cloud obtained from an RGB-D camera as a state input,and used two deep Q-network algorithms to fit the grasping and pushing strategies separately.Through the reasonable design of the reward function,the coordination between grasping and pushing was learnt to change the distribution of objects through pushing actions and provide favorable conditions for grasping.Furthermore,to address the slow training speed resulting from the complexity of action space,a normal mask-based action space optimization strategy was proposed by limiting the exploration space of grasping and pushing actions based on prior knowledge of estimated normal direction.Finally,experiments were conducted with UR5 robot in the real world to verify the effectiveness of the proposed cooperative grasping strategy.

Key words: deep reinforcement learning, robot grasping, 3D point cloud, multi-action collaboration

摘要： 为解决机器人自主抓取领域中,抓取区域内存在明显遮挡且相互堆叠现象导致的抓取成功率不高的问题,提出一种多视角下基于深度强化学习的“抓取”和“推动”协同抓取策略。该策略利用基于RGB-D相机获取的三维点云作为状态输入,通过两个深度Q网络算法分别拟合抓取和推动策略,并通过奖励函数的合理设计,学习抓取和推动的相互配合,旨在用推动动作改变物体的分布,为抓取提供有利条件。在此基础上,针对抓取动作空间的复杂性导致算法抓取成功率不高的问题,提出了一种基于法线掩码的动作空间优化策略,通过基于估计法线的先验知识对抓取和推动动作的探索空间进行限制,从而提升了算法的性能。最后,利用UR5机器人在真实世界中进行协同抓取策略实验,验证了所提出协同抓取策略的有效性。

关键词: 深度强化学习, 机器人抓取, 三维点云, 多动作协同