用语言化动作表示解决机器人控制中的尺度差异问题
Bridging Scale Discrepancies in Robotic Control via Language-Based Action Representations
- 提出语义化的动作表示，忽略数值尺度，只关注方向性
- 在两个基准上显著提升多任务泛化与迁移能力
- 适合做跨平台机器人预训练的研究者参考
近期端到端机器人操作研究越来越多地采用受大语言模型启发的架构，以实现鲁棒的操作能力。然而，由于不同机器人平台和任务间动作命令存在显著的数值差异，导致动作数据分布严重偏移，阻碍了预训练知识的有效迁移。为解决这一问题，我们提出一种语义基础的语言动作表示，用于标准化动作以实现高效预训练。与传统对数值尺度敏感的离散动作表示不同，该运动表示专门忽略数值尺度影响，强调方向性，从而缓解分布偏移，获得更具泛化性的预训练表示。此外，使用该表示可缩小动作标记与标准词汇标记之间的特征距离，缓解模态差距。在两个基准上的多任务实验表明，该方法显著提升了机器人操作任务中的泛化性能和可迁移性。
原文摘要 · Abstract (English)
Recent end-to-end robotic manipulation research increasingly adopts architectures inspired by large language models to enable robust manipulation. However, a critical challenge arises from severe distribution shifts between robotic action data, primarily due to substantial numerical variations in action commands across diverse robotic platforms and tasks, hindering the effective transfer of pretrained knowledge. To address this limitation, we propose a semantically grounded linguistic representation to normalize actions for efficient pretraining. Unlike conventional discretized action representations that are sensitive to numerical scales, the motion representation specifically disregards numeric scale effects, emphasizing directionality instead. This abstraction mitigates distribution shifts, yielding a more generalizable pretraining representation. Moreover, using the motion representation narrows the feature distance between action tokens and standard vocabulary tokens, mitigating modality gaps. Multi-task experiments on two benchmarks demonstrate that the proposed method significantly improves generalization performance and transferability in robotic manipulation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。