首个专为钢构装配感知设计的混合数据集,提升机器人识别精度。
ISC-Perception: A Hybrid Computer Vision Dataset for Object Detection in Novel Steel Assembly
- 融合渲染、游戏引擎与真实照片,自动生成带标签数据。
- 训练模型在测试集上达[email protected] 0.943,显著优于单一数据源。
- 适合建筑机器人、工业检测领域研究者快速开发检测系统。
互锁钢连接(ISC)系统搭配机器人可加速钢框架装配并提升工人安全,但其可靠感知受限于缺乏专用图像数据集——现场拍摄面临物流难、安全与隐私问题。为此,我们提出ISC-Perception,首个专为ISC部件检测设计的混合数据集,融合程序化生成的CAD图像、游戏引擎渲染的逼真场景及少量人工筛选的真实照片,实现合成部分的全自动标注。我们详细记录了所有人力投入:从仿真引擎配置、场景搭建、资产准备到后处理脚本与质检,生成1万张图像仅需30.5小时,相较人工标注每图60秒、总计166.7小时,效率提升81.7%。手动基准测试中,一张含5个构件的图像耗时60秒(最长80秒)。基于该数据集训练的检测器在验证集上达到[email protected] 0.756,显著优于纯合成或纯逼真数据训练的模型。在1200帧的基准测试中,[email protected]/mAP@[0.50:0.95]分别为0.943/0.823。ISC-Perception填补了建筑机器人感知的数据空白,支持定制检测器快速开发,可应需求免费用于科研与产业应用。
原文摘要 · Abstract (English)
The Intermeshed Steel Connection (ISC) system, when paired with robotic manipulators, can accelerate steel-frame assembly and improve worker safety by eliminating manual assembly. Dependable perception is one of the initial stages for ISC-aware robots. However, this is hampered by the absence of a dedicated image corpus, as collecting photographs on active construction sites is logistically difficult and raises safety and privacy concerns. In response, we introduce ISC-Perception, the first hybrid dataset expressly designed for ISC component detection. It blends procedurally rendered CAD images, game-engine photorealistic scenes, and a limited, curated set of real photographs, enabling fully automatic labelling of the synthetic portion. We explicitly account for all human effort to produce the dataset, including simulation engine and scene setup, asset preparation, post-processing scripts and quality checks; our total human time to generate a 10,000-image dataset was 30.5,h versus 166.7,h for manual labelling at 60,s per image (-81.7%). A manual pilot on a representative image with five instances of ISC members took 60,s (maximum 80,s), anchoring the manual baseline. Detectors trained on ISC-Perception achieved a mean Average Precision at IoU 0.50 of 0.756, substantially surpassing models trained on synthetic-only or photorealistic-only data. On a 1,200-frame bench test, we report [email protected]/mAP@[0.50:0.95] of 0.943/0.823. By bridging the data gap for construction-robotics perception, ISC-Perception facilitates rapid development of custom object detectors and is freely available for research and industrial use upon request.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。