用合成数据训练视觉系统,高效识别装配步骤
A Synthetic-Driven Vision System for Assembly Step Recognition

- 基于CAD和步骤描述自动生成逼真装配序列
- 真实场景下达到92.4%准确率,提升超46%
- 无需实拍数据,适合快速部署到工业产线
工业装配质量控制至关重要,实时监控可预防昂贵缺陷并保障生产可靠性。基于视觉的自动检测为实时监控提供了有效方案,但因工业部件与工艺特殊,模型训练通常依赖特定任务的真实数据,采集与标注成本高、耗时长。本文提出一个系统,能自动生成逼真装配序列,并利用合成数据训练实时检测模型。该系统可在一小时内完成部署,仅需CAD模型和简单步骤说明。针对实际挑战,系统集成物理驱动的运动生成模块以模拟不同操作者差异,采用领域随机渲染应对环境复杂性,结合基于目标检测的步骤识别模块实现稳健的仿真到现实迁移。在真实装配任务中达到92.4%准确率,较基线分别提升46.7%、15.8%和61.2%。整体方案无需昂贵的真实数据收集与标注,在真实工业任务中验证了有效性。
原文摘要 · Abstract (English)
Quality control in industrial assembly is essential, and real-time monitoring of the assembly process is crucial for preventing costly defects and ensuring production reliability. Vision-based automated inspection offers a powerful solution for such real-time monitoring. However, due to the specialized industrial components and processes, training these models typically relies on task-specific real-world data, which is costly and labor-intensive to collect and annotate. In this paper, we propose a system that automatically generates realistic assembly sequences and further trains real-time inspection models using the synthetic data. It can be efficiently applied to a given task within an hour, requiring only CAD models and simple step descriptions. Focusing on practical challenges, our system integrates a physics-based motion generation module to capture the variance of different human assembly, designs domain-randomized rendering to deal with the environmental complexity and variation, and employs an object-detection-based step recognition module for robust sim-to-real transfer, leading to 92.4% accuracy on a real-world assembly case with 46.7%, 15.8% and 61.2% performance improvement, respectively. Overall, our system provides a practical solution for industrial assembly inspection without requiring expensive real-world data collection and annotation, with the effectiveness validated on real industrial assembly tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。