arXiv:2608.29601cs.ROcs.CV2026-08被引 2

构建触觉智能体操控新范式,打通触觉感知到动作执行的闭环。

$\mathcal{N}_0$-Foundation: Towards the Age of Tactile Intelligence

论文配图:$\mathcal{N}_0$-Foundation: Towards the Age of Tactile Intelligence
图 1 · 摘自论文原文
  • 自研触觉传感器与数据采集系统,实现多形态机器人同步视觉触觉数据采集。
  • 构建超大规模触觉数据集NeoData(3万小时),支持柔性物体操控等复杂任务。
  • 提出可迁移触觉表征模型NeoForce,适合研究触觉智能与具身智能的学者。

我们提出$_N_0$-Foundation,一种面向触觉赋能的具身操控范式,整合了触觉传感硬件、大规模多模态数据、触觉表征学习与标准化评估体系。首先,我们构建了可扩展的数据采集基础设施,包括基于视觉的触觉传感器、触觉通用操控接口(UMI)以及支持机器人本体与UMI演示的同步视听触觉数据采集系统。基于该系统,我们构建了包含超过30000小时同步视觉与触觉演示的NeoData数据集,涵盖六种机器人本体、450项任务,以及通过真实机器人遥操作与UMI演示收集的数十亿对RGB与触觉帧。为促进开放研究,我们进一步发布5000小时开源子集OpenNeoData。该数据集弥补了现有操控数据集在柔性物体操作、精密装配、精细力控和持续表面交互方面的关键不足。利用大规模异构触觉测量数据,我们提出NeoForce模型,可在不同触觉传感器设计间学习可迁移的触觉表征。为系统评估基于本基础设施、数据集与触觉表征构建的具身模型,我们进一步提出综合性基准,结合真实世界中的NeoReal套件与模拟环境中的NeoSim套件,实现标准化评估。跨两个套件的实验表明,策略性能受益于物理接触状态,而非触觉信号的设备特定外观。我们发布数据集、表征模型与评估基准,旨在支持未来触觉赋能具身操控的研究。

原文摘要 · Abstract (English)

We present $\mathcal{N}_0$-Foundation, a paradigm for tactile-enabled embodied manipulation, which integrates tactile sensing hardware, large-scale multimodal data, tactile representation learning, and standardized evaluation. First, we engineer the infrastructure for scalable data collection, including a vision-based tactile sensor, a tactile Universal Manipulation Interface (UMI), and a synchronized visuo-tactile data collection system supporting both robot embodiments and UMI-based demonstrations. Leveraging this infrastructure, we construct NeoData, which contains more than 30000 hours of synchronized visual and tactile demonstrations, spanning six embodiments, 450 tasks, and billions of paired RGB and tactile frames collected through a mixture of real-robot teleoperation and UMI-based demonstrations. To facilitate open research, we further release OpenNeoData, a 5000-hour open-source subset of NeoData. The dataset addresses a central limitation of existing manipulation corpora, critical for deformable-object manipulation, precise assembly, delicate force control, and sustained surface interaction. Capitalizing on the large-scale, heterogeneous tactile measurements, we propose NeoForce, a visuo-tactile representation model that learn transferable tactile representations across different sensor designs. To enable systematic evaluation of tactile embodied models built upon our infrastructure, datasets and tactile representations, we further propose a comprehensive benchmark, which combines the real-world NeoReal suite and the simulated NeoSim suite for standardized evaluation. Experiments across both suites show that policies benefit from the physical contact state rather than from the device-specific appearance of the tactile signal. We release the dataset, the representation, and the benchmark, aiming at supporting future work on tactile-enabled embodied manipulation.

触觉智能具身智能多模态数据机器人操控

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。