提出有序动作分词方法,让机器人用更少令牌实现高效推理。
OAT: Ordered Action Tokenization
- 用带记忆单元的Transformer+量化+排序训练,生成有顺序的动作令牌
- 在20多个任务中比之前方法更优,推理时可自由调节精度和速度
- 适合需要灵活实时决策的机器人控制场景
自回归策略通过离散抽象、令牌级推理和灵活推理,为可扩展机器人学习提供了有力基础。然而,将自回归建模应用于连续机器人动作,需有效的动作分词方案。现有方法或依赖分析性离散化,导致令牌序列过长;或采用无结构的可学习潜在分词器,难以兼容下一步令牌预测。本文提出动作分词的三个理想特性:高压缩率、完全可解码性、左到右因果有序的令牌空间,并引入有序动作分词(OAT)——一种满足全部特性的可学习分词器。OAT利用Transformer与寄存器、有限标量量化及诱导排序的训练机制,将动作块离散化为有序令牌序列。该令牌空间天然适配自回归生成,支持前缀式解码,实现推理成本与动作保真度之间的任意权衡。在超过20项任务、四个仿真基准及真实世界设置中,配备OAT的自回归策略持续优于先前分词方案与扩散基线,同时提供显著更高的推理灵活性。
原文摘要 · Abstract (English)
Autoregressive policies offer a compelling foundation for scalable robot learning by enabling discrete abstraction, token-level reasoning, and flexible inference. However, applying autoregressive modeling to continuous robot actions requires an effective action tokenization scheme. Existing approaches either rely on analytical discretization methods that produce prohibitively long token sequences, or learned latent tokenizers that lack structure, limiting their compatibility with next-token prediction. In this work, we identify three desiderata for action tokenization - high compression, total decodability, and a left-to-right causally ordered token space - and introduce Ordered Action Tokenization (OAT), a learned action tokenizer that satisfies all three. OAT discretizes action chunks into an ordered sequence of tokens using transformer with registers, finite scalar quantization, and ordering-inducing training mechanisms. The resulting token space aligns naturally with autoregressive generation and enables prefix-based detokenization, yielding an anytime trade-off between inference cost and action fidelity. Across more than 20 tasks spanning four simulation benchmarks and real-world settings, autoregressive policies equipped with OAT consistently outperform prior tokenization schemes and diffusion-based baselines, while offering significantly greater flexibility at inference time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。