无需标注数据,通过实时3D数字孪生实现工业缺陷零样本检测
Zero-Shot Multi-Criteria Visual Quality Inspection for Semi-Controlled Industrial Environments via Real-Time 3D Digital Twin Simulation
- 用已知3D模型和视觉识别构建实时数字孪生,实现无姿态依赖的缺陷对比
- 在半受控环境下实现最高63.3%的交并比,仅靠距离测量即可完成检测
- 适用于汽车电机等场景,适合想降低数据依赖的工业质检研究者
早期视觉质量检测对实现零缺陷制造和减少生产浪费至关重要。然而,复杂且需大量数据的检测系统限制了其在半受控工业环境中的应用。本文提出一种无姿态依赖、零样本的质量检测框架,将真实场景与实时生成的RGB-D空间数字孪生(Digital Twin, DT)进行对比。通过已知计算机辅助设计(CAD)模型的物体检测与位姿估计,实现对工业场景的语义描述,并高效渲染实时数字孪生。我们评估了多种实时多模态RGB-D数字孪生创建工具,并追踪其计算资源消耗。此外,提出可扩展的分层标注策略,统一位姿标注与逻辑/结构缺陷标注。基于轴向磁通电机的汽车应用场景验证了该框架的有效性。结果表明,在半受控工业条件下,即使使用简单距离测量,也能达到最高63.3%的交并比(IoU),相比真实标注掩码表现优异。研究为动态制造环境中通用、低数据需求的缺陷检测方法奠定了基础。
原文摘要 · Abstract (English)
Early-stage visual quality inspection is vital for achieving Zero-Defect Manufacturing and minimizing production waste in modern industrial environments. However, the complexity of robust visual inspection systems and their extensive data requirements hinder widespread adoption in semi-controlled industrial settings. In this context, we propose a pose-agnostic, zero-shot quality inspection framework that compares real scenes against real-time Digital Twins (DT) in the RGB-D space. Our approach enables efficient real-time DT rendering by semantically describing industrial scenes through object detection and pose estimation of known Computer-Aided Design models. We benchmark tools for real-time, multimodal RGB-D DT creation while tracking consumption of computational resources. Additionally, we provide an extensible and hierarchical annotation strategy for multi-criteria defect detection, unifying pose labelling with logical and structural defect annotations. Based on an automotive use case featuring the quality inspection of an axial flux motor, we demonstrate the effectiveness of our framework. Our results demonstrate detection performace, achieving intersection-over-union (IoU) scores of up to 63.3% compared to ground-truth masks, even if using simple distance measurements under semi-controlled industrial conditions. Our findings lay the groundwork for future research on generalizable, low-data defect detection methods in dynamic manufacturing settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。