无需训练的机器人钢筋绑扎系统，实测精度超越传统方法。
OpenTie: Open-vocabulary Sequential Rebar Tying System
- 用图像转点云+开放词汇检测，免训练完成钢筋识别
- 真实场景下序列绑扎测试中，精度优于YOLO等训练模型
- 适用于水平与垂直钢筋作业，具备工地商用潜力
建筑工地中的机器人作业因应对复杂任务的能力而备受关注，尤其在钢筋相关场景中。现有多数产品与研究依赖大量数据收集与模型训练。为填补这一空白，我们提出OpenTie——一种无需训练的3D钢筋绑扎框架，结合RGB到点云生成与开放词汇钢筋检测，在真实世界测试中实现高精度。通过双目相机与机械臂实现系统部署，并采用自研后处理流程优化图像到点云生成，再以提示驱动的物体检测方法进行识别。该流程无需训练，且在真实场景下的连续钢筋绑扎测试中表现优于基于YOLO的训练型检测方法。系统可灵活适应水平与垂直钢筋绑扎任务，具备实际工地应用与商业化前景。
原文摘要 · Abstract (English)
Robotic practices on the construction site emerge as an attention-attracting manner owing to their capability of tackling complex challenges, especially in the rebar-involved scenarios. Most of existing products and research are mainly focused on the collection of large amounts of data with model training demands. To fulfill this gap, we propose OpenTie, a 3D training-free rebar tying framework utilizing a RGB-to-point-cloud generation and an open-vocabulary rebar detection on the real-world test. We implement the OpenTie via a robotic arm with a binocular camera and guarantee a high accuracy by applying the prompt-based object detection method on the image filtered by our proposed post-processing procedure for the image-to-point-cloud generation framework. Our pipeline requires no training efforts and outperforms the training-based object detection, i.e., YOLO-based method, with the verification on the real-world sequential rebar tying test. The system is flexible for horizontal and vertical rebar tying tasks and holds the potential application to the real construction site with possibility of commercialization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。