Abstract: Existing vision-language models focus too much on object category semantics,and they ignore the fine-grained perception of local spatial defect areas.To solve these problems,a zero-shot defect detection method was proposed based on spatial semantic guidance.The guidance network was designed to extract semantic distribution features of images.These distribution features were added to the visual encoding network of the vision-language model.Highly universal learnable text prompts were designed for both normal and defect states.The corresponding text embedding was extracted by the designed text encoding network.The defect heat map was predicted based on cosine similarity between text embedding and multiple visual features.The proposed method achieved pixel level defect detection accuracy of 88.5%,95.3%,97.0% and 91.6% on the MVTec,VisA,MPDD and BTAD datasets respectively.Experimental results showed that the proposed method had strong zero-shot defect detection performance.

Key words: zero-shot defect detection, vision-language model, learnable prompt, semantic guidance

摘要： 针对现有视觉语言模型过多关注物体类别语义,忽略局部空间缺陷区域的细粒度感知问题,提出基于空间语义引导的零样本缺陷检测方法。设计空间语义引导网络提取图像语义分布特征,并将其添加到视觉语言模型中的视觉编码网络。针对正常和缺陷状态设计通用性较强的可学习文本提示,由设计的文本编码网络提取对应的文本嵌入,并与多阶段视觉特征计算余弦相似度,进而预测缺陷区域热图。所提缺陷检测模型在MVTec、VisA、MPDD、BTAD四个数据集上分别获得了88.5%、95.3%、97.0%、91.6%的像素级缺陷检测准确率。实验结果表明所提方法具有较强的零样本缺陷检测性能。

关键词: 零样本缺陷检测, 视觉语言模型, 可学习提示, 语义引导