用音频压缩技术改造机器人动作编码,提升控制精度与效率。
NAC: Neural Action Codec for Vision-Language-Action Models

- 将动作轨迹视为一维信号,用多尺度向量量化生成紧凑动作码本。
- 在多个数据集上重建误差更低,成功率显著高于现有方法。
- 适合需要高精度动作序列建模的机器人学习任务。
视觉-语言-动作(VLA)模型依赖离散动作分词器连接连续机器人控制与自回归序列建模,但现有分词器常在压缩率、延迟与下游性能间权衡。本文借鉴神经音频编解码器——基于卷积编码器-解码器与残差向量量化(RVQ)的架构——提出神经动作编解码器(NAC),将短时机器人动作轨迹视为多通道一维信号,采用多尺度RVQGAN进行压缩。发现音频特有的梅尔频谱损失不适用于运动学信号;通过替换为时域与非梅尔频谱重建损失,无需大幅修改架构即可实现高保真动作自编码。NAC通过偏移码本构建紧凑且有序的动作词空间,使标准自回归策略可处理短而结构化的序列。同时,采用类似Vocos的解码器搭配逆短时傅里叶变换(ISTFT)头与对抗判别器,恢复出平滑、细节丰富的轨迹。在LIBERO-10、RoboMimic及一系列真实世界操作任务中,NAC在相当或更优的压缩率下,重建误差更低,成功率达更高,优于分箱法、FAST及先前基于VQ的分词器。结果表明,复用神经音频编解码器为现代VLA中的学习型动作分词提供了强大且实用的基础。
原文摘要 · Abstract (English)
Vision-language-action (VLA) models rely on discrete action tokenizers to bridge continuous robot control and autoregressive sequence modeling, yet existing tokenizers often trade off between compression, latency, and downstream performance. We revisit this design through the lens of neural audio codecs-convolutional encoder-decoder architectures with residual vector quantization that serve as the standard front end for audio foundation models. Motivated by their success, we introduce the Neural Action Codec (NAC), which treats short robot action trajectories as multi-channel 1D signals and compresses them using a multi-scale RVQGAN architecture. We observe that audio-specific mel-spectrogram objectives are ill-suited for kinematic signals; however, by replacing them with simple time-domain and non-mel spectral reconstruction losses, audio-codec-style models can autoencode actions with high fidelity without substantial architectural changes. NAC provides a compact, ordered token space via offset codebooks, enabling standard autoregressive policies to operate over short, structured sequences. Meanwhile, a Vocos-style decoder with an ISTFT head and adversarial discriminators recovers smooth, detailed trajectories. Across LIBERO-10, RoboMimic, and a suite of real-world manipulation tasks, NAC achieves lower reconstruction error and higher success rates than binning, FAST, and prior VQ-based tokenizers at comparable or better compression rates. These results demonstrate that repurposed neural audio codecs offer a strong, practical backbone for learned action tokenization in modern VLAs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。