arXiv:2605.27886cs.RO2026-05被引 3

用视觉触觉语言闭环反馈,让机器人更轻柔地抓取物品

Tabero: Learning Gentle Manipulation with Closed-Loop Force Feedback from Vision, Touch, and Language

论文配图:Tabero: Learning Gentle Manipulation with Closed-Loop Force Feedback from Vision, Touch, and Language
图 1 · 摘自论文原文
  • 设计多模态数据生成流程,解决触觉数据稀缺问题
  • 实现70%以上握力降低,同时保持任务成功率
  • 适合研究人机交互与柔顺操作的开发者

触觉感知对机器人实现类人柔顺操作至关重要。然而,现有视觉-语言-动作(VLA)模型因缺乏对齐的视觉-触觉-语言数据,且缺少有效的闭环力反馈机制,难以有效利用触觉信息完成柔顺操作。为此,我们提出Tabero,一个面向语言驱动柔顺操作的基准与模型套件,要求精细的接触力感知。首先,通过复用开源机器人操作轨迹,构建数据高效的数据生成管道,生成多样化的视觉-触觉-语言任务,并建立多维度评估协议,同时衡量任务成功率与物理交互质量。其次,提出Tabero-VTLA架构,采用解耦的力-位置指令接口,由固定混合控制器实时执行,实现力感知操作。在Tabero基准上评估显示,该模型在温和指令下平均握力降低超过70%,同时保持高任务成功率,证明其能基于多模态经验调节交互力。代码已公开于https://github.com/NathanWu7/Tabero。

原文摘要 · Abstract (English)

Tactile sensing is essential for robots to achieve human-like gentle manipulation. However, existing Vision-Language-Action (VLA) models struggle to exploit tactile feedback for gentle manipulation due to scarce aligned vision-tactile-language data and the lack of effective closed-loop force feedback mechanisms. To address these challenges, we introduce Tabero, a benchmark and model suite for gentle, language-conditioned robotic manipulation that demands fine-grained contact force perception. First, the Tabero benchmark addresses the scarcity of tactile data by presenting a data-efficient pipeline that repurposes open-source robot manipulation trajectories to generate diverse vision-tactile-language tasks, and establishes a multidimensional evaluation protocol that measures task success alongside physical interaction quality. Second, we propose Tabero-VTLA, an architecture with a decoupled force-position command interface; the resulting force-position commands are executed by a fixed hybrid controller to enable real-time, force-aware manipulation. Evaluated on Tabero, our model maintains high task success while reducing average grip force by over 70\% under gentle instructions, demonstrating its ability to modulate interaction forces based on multimodal experience. Our code is publicly available at https://github.com/NathanWu7/Tabero.

柔顺操作触觉反馈多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。