Abstract: To improve the efficiency of manual assembly process,a gesture recognition and error prevention method to guarantee process sequence was proposed.An Augmented Reality (AR) assembly assistant system controlled by vision recognized signal was presented to guide error correction,which reduced the time cost of interaction and rework.An example assembly video was captured by an ego-centric camera,and time measurement method was used to analyze the assembly operation time and sequence.The assembly process simulation was performed to construct the assembly guidance information model.Hand region on each frame of sampled video was extracted and then processed into geometric features.Gesture recognition constraints for each operation step was proposed based on probability distribution fitting.After adding the assembly time and sequential constraints,the recognized gesture features were further processed to assembly sequence signals that update the user's assembly progress and control the display of AR guidance information.The proposed methods were verified by a computer motherboard assembly case study,and the results showed an increment on assembly efficiency and one-time completion rate.

Key words: gesture recognition, Poka-Yoke, sequential control, manual assembly, augmented reality

摘要： 为提高手工装配生产过程的效率,提出一种保证过程时序性的手势视觉识别与防错方法,通过识别信号控制增强现实装配引导系统,减少装配过程中检错和返工消耗的时间。在第一视角下采集手工装配过程的样本视频,用预定时间方法分析单步操作的标准手势和时间序列。在虚拟装配环境中建立了装配操作序列仿真引导模型。用肤色检测获取装配图像的手部区域,定义了区域几何特征并进行概率分布拟合,提取装配手势特征及其识别条件。基于装配动作的时序约束,建立了时序锁定的装配手势识别方法,进而判断装配状态,触发推送、隐藏或更新引导信息。计算机主板装配实例表明,所提出的智能防错方法提高了装配序列的执行效率和一次完成率。

关键词: 手势识别, Poka-Yoke, 时序控制, 手工装配, 增强现实