提出有序动作分词方法,让机器人控制更高效灵活
Ordered Action Tokens for Visuomotor Policy Learning

- 用带记忆单元的Transformer实现有顺序的动作离散化
- 早期令牌控制粗略动作,后期细化细节,支持随时调整精度
- 适配多种策略模型,在仿真和真实场景中表现稳定
动作分词将连续机器人动作片段映射为离散令牌,是现代视觉-运动策略的重要接口。现有方法或依赖分析式离散化导致令牌序列过长,或使用无结构的可学习分词器,影响下游策略兼容性。本文提出有序动作分词(OAT),满足高压缩、完全可解码和有序令牌空间三大需求。OAT利用Transformer与寄存器、有限标量量化及排序诱导训练机制,将动作片段离散为有序令牌序列。通过训练每个令牌前缀可解码为有效动作片段,使早期令牌承载粗控信息,后期令牌用于精修残差细节,实现推理成本与动作保真度的任意权衡。在两种主流动作令牌应用中验证:自回归策略生成令牌进行控制,以及令牌协同训练策略通过令牌损失塑造流模型动作专家所用的视觉-语言上下文。在三种策略骨干和超过60个任务(覆盖五个仿真基准和真实场景)中,OAT始终表现优异,并在推理阶段提供显著灵活性。
原文摘要 · Abstract (English)
Action tokenization maps continuous robot action chunks to discrete tokens and has become an important interface for modern visuomotor policies. Existing approaches either rely on analytical discretization methods that produce prohibitively long token sequences or learned latent tokenizers that lack structure, limiting their compatibility with downstream policies. In this work, we identify three desiderata for action tokenization - high compression, total decodability, and an ordered token space - and introduce Ordered Action Tokenization (OAT), a learned action tokenizer that satisfies all three. OAT discretizes action chunks into an ordered sequence of tokens using a transformer with registers, finite scalar quantization, and ordering-inducing training mechanisms. By training each token prefix to decode into a valid action chunk, OAT places coarse control information in early tokens and uses later tokens to refine residual detail, yielding an anytime tradeoff between inference cost and action fidelity. We validate OAT in two prevailing uses of action tokens: autoregressive policies that generate tokens for control, and token co-training policies that use token losses to shape the vision-language model context consumed by a flow-based action expert. Across three policy backbones and more than 60 tasks spanning five simulation benchmarks and real-world settings, OAT consistently delivers strong policy performance while offering significantly greater flexibility at inference time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。