Abstract: To replace the manual inspection of connecting states between wiring harness and connectors in the assembly process of aerospace products,a smart inspection method assisted by Augmented Reality (AR) glasses for assembly states of connectors in wiring harness was proposed.The smart-inspection automatically conducted three major processes of pin-hole detection,pin-hole sequencing and mismatched pin-hole visualization.For pin-hole detection,the detection model combining the Feature Pyramid Networks (FPN) architecture and the Bi-directional Long Short-Term Memory (BiLSTM) component effectively located pin-holes' location in images and recognized their actual assembly states.These detected pin-holes were clustered into different annuluses and numbered according to their polar angle in the operation of pin-hole sequencing.Traversing all numbered pin-holes in order,mismatched pin-holes that were missing with wires or contain extra mismatched wires were identified by comparing the ground truth of assembly results and actual assembly states.AR glasses presented the wireframe image which could highlight these mismatched pin-holes for visualization.For eleven different connectors,the pin-hole detection model achieved a competitive performance with the figure for accuracy up to 99.00% in mean Average Precision (mAP) which was the metric in PASCAL VOC dataset.Meanwhile,the pin-hole sequencing method even against the negative effect of image deformation in some extends.

Key words: augmented reality glasses, assisted assembly, deep learning, harness connector, pin-hole detection, pin-hole sequencing

摘要： 为替代航空航天产品装配过程中人工判定线缆连接器组装结果的操作,提出一种增强现实(AR)眼镜辅助的线缆连接器装配状态智能检错方法。该智能检错方法包括自动检测连接器孔位、分布排序和可视化不匹配孔位三个环节。面向孔位检测,融合了特征金字塔网络架构与双向长短期记忆网络结构的孔位检测网络可以有效定位图像中孔位位置并识别其真实安装状态;在孔位分布排序流程中,对检测到的孔位进行聚类划分至不同圆环上,排序每层圆环上孔位的极角以获得编号;顺序遍历所有编号的孔位,比对每个孔位的理想安装结果和真实安装状态可得知错装导线孔位与漏装导线的孔位,并使用AR眼镜可视化型谱图高亮这些不匹配孔位。采用PASCAL VOC数据集的评价指标,以11种连接器的检错结果为例进行验证,提出的孔位检测网络的平均精度均值高达99.00%;同时孔位分布排序算法针对图像形变情况有较强的鲁棒性。

关键词: 增强现实眼镜, 辅助装配, 深度学习, 线缆连接器, 孔位检测, 孔位排序