提出FAST方法,用频域变换提升机器人动作的高效离散化。
FAST: Efficient Action Tokenization for Vision-Language-Action Models

- 基于离散余弦变换实现动作序列的频域压缩式分词。
- 在高频高精度任务中显著优于传统逐维分箱法,支持10000小时数据训练。
- 可适配多种机器人动作空间,适合构建通用动作分词器。
自回归序列模型(如基于Transformer的视觉-语言-动作模型)在捕捉复杂且可泛化的机器人行为方面极为有效。然而,这类模型需对连续动作信号进行分词,决定模型预测的离散符号如何映射为连续机器人动作。现有基于每维度、每时间步简单分箱的方法,在从高频机器人数据学习精细操作技能时表现不佳。为此,我们提出一种基于压缩思想的动作分词方案,即频率域动作序列分词(FAST),该方法利用离散余弦变换实现动作分词。基于FAST,我们发布了FAST+,一个在100万条真实机器人动作轨迹上训练的通用动作分词器,可作为黑盒工具应用于多种动作空间与控制频率的机器人序列。结合pi0 VLA模型,该方法可实现10000小时数据训练,性能媲美扩散型VLA,同时将训练时间减少高达5倍。
原文摘要 · Abstract (English)
Autoregressive sequence models, such as Transformer-based vision-language action (VLA) policies, can be tremendously effective for capturing complex and generalizable robotic behaviors. However, such models require us to choose a tokenization of our continuous action signals, which determines how the discrete symbols predicted by the model map to continuous robot actions. We find that current approaches for robot action tokenization, based on simple per-dimension, per-timestep binning schemes, typically perform poorly when learning dexterous skills from high-frequency robot data. To address this challenge, we propose a new compression-based tokenization scheme for robot actions, based on the discrete cosine transform. Our tokenization approach, Frequency-space Action Sequence Tokenization (FAST), enables us to train autoregressive VLAs for highly dexterous and high-frequency tasks where standard discretization methods fail completely. Based on FAST, we release FAST+, a universal robot action tokenizer, trained on 1M real robot action trajectories. It can be used as a black-box tokenizer for a wide range of robot action sequences, with diverse action spaces and control frequencies. Finally, we show that, when combined with the pi0 VLA, our method can scale to training on 10k hours of robot data and match the performance of diffusion VLAs, while reducing training time by up to 5x.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。