用统一隐空间让机器人跨手型学会灵巧操作
Cross-Hand Latent Representation for Vision-Language-Action Models
- 构建跨多种机械手的统一动作隐空间，支持不同机器人共享训练数据
- 在多个机械手上表现优于直接使用关节空间的基线模型
- 适合需要快速适配新机械手的灵巧操作研究与应用
灵巧操作是实现真实世界机器人自主的关键，如同人类日常活动依赖双手协调。人类通过视觉、听觉和语言提示等多种感知完成灵巧动作，启发了基于视觉和语言引导的机器人操作系统。然而，训练可靠的视觉-语言-动作（VLA）模型需要大量来自多种机械手的操作示范数据。随着新型灵巧机械手快速出现，为每种手收集数据成本高昂且不切实际，亟需可扩展的跨机体学习方法。本文提出XL-VLA，一种集成统一隐动作空间的视觉-语言-动作框架，该空间在多种灵巧机械手中共享。此机体无关的隐空间可直接接入标准VLA架构，实现跨机体训练无缝衔接，并高效复用已有及新采集的数据。实验表明，XL-VLA在多个机械手上持续优于在原始关节空间运行的基线模型，验证了其在可扩展跨机体灵巧操作中的有效性。
原文摘要 · Abstract (English)
Dexterous manipulation is essential for real-world robot autonomy, mirroring the central role of human hand coordination in daily activity. Humans rely on rich multimodal perception--vision, sound, and language-guided intent--to perform dexterous actions, motivating vision-based, language-conditioned manipulation systems for robots. However, training reliable vision-language-action (VLA) models for dexterous manipulation requires large-scale demonstrations across many robotic hands. In addition, as new dexterous embodiments appear rapidly, collecting data for each becomes costly and impractical, creating a need for scalable cross-embodiment learning. We introduce XL-VLA, a vision-language-action framework integrated with a unified latent action space shared across diverse dexterous hands. This embodiment-invariant latent space is directly pluggable into standard VLA architectures, enabling seamless cross-embodiment training and efficient reuse of both existing and newly collected data. Experimental results demonstrate that XL-VLA consistently outperforms baseline VLA models operating in raw joint spaces, establishing it as an effective solution for scalable cross-embodiment dexterous manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。