无需训练模型,通过机器人交互实时分割未知物体,提升抓取成功率。
rt-RISeg: Real-Time Model-Free Robot Interactive Segmentation for Active Instance-Level Object Understanding
- 利用机器人随机动作产生的运动信息,自动生成物体分割图
- 相比顶尖方法,分割准确率提升27.5%,且无需等待动作完成
- 适合需要快速理解新环境的自主机器人系统
在新环境中成功执行灵巧操作任务(如抓取)依赖于对未知物体的精准分割。以往的未见物体实例分割(UOIS)方法基于大规模数据集训练模型,易因静态视觉特征过拟合,导致分布外场景泛化能力差。为此,我们重新思考UOIS任务——视觉本质上是交互式且随时间演变的。提出一种实时交互感知框架rt-RISeg,通过机器人交互与设计的体帧不变特征(BFIF)持续分割未知物体。我们发现,随机采样的体帧相对旋转与线速度可无须学习模型即可识别物体。该全自包含流水线在每次机器人交互中生成并更新分割掩码,无需等待动作结束。实验表明,rt-RISeg平均分割准确率比现有最优方法高出27.5%。此外,尽管rt-RISeg为独立框架,其自动生成的分割掩码可作为提示输入视觉基础模型,显著提升性能。
原文摘要 · Abstract (English)
Successful execution of dexterous robotic manipulation tasks in new environments, such as grasping, depends on the ability to proficiently segment unseen objects from the background and other objects. Previous works in unseen object instance segmentation (UOIS) train models on large-scale datasets, which often leads to overfitting on static visual features. This dependency results in poor generalization performance when confronted with out-of-distribution scenarios. To address this limitation, we rethink the task of UOIS based on the principle that vision is inherently interactive and occurs over time. We propose a novel real-time interactive perception framework, rt-RISeg, that continuously segments unseen objects by robot interactions and analysis of a designed body frame-invariant feature (BFIF). We demonstrate that the relative rotational and linear velocities of randomly sampled body frames, resulting from selected robot interactions, can be used to identify objects without any learned segmentation model. This fully self-contained segmentation pipeline generates and updates object segmentation masks throughout each robot interaction without the need to wait for an action to finish. We showcase the effectiveness of our proposed interactive perception method by achieving an average object segmentation accuracy rate 27.5% greater than state-of-the-art UOIS methods. Furthermore, although rt-RISeg is a standalone framework, we show that the autonomously generated segmentation masks can be used as prompts to vision foundation models for significantly improved performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。