arXiv:2606.30113cs.ROcs.AI2026-06

让机器人动作解码更智能:根据当前状态调整动作解析,提升操作成功率。

SA-VLA: State-aware tokenizer for improving Vision-Language-Action Models' performance

论文配图:SA-VLA: State-aware tokenizer for improving Vision-Language-Action Models' performance
图 1 · 摘自论文原文
  • 用机器人状态动态调节动作解码,突破固定动作原型的限制。
  • 在12个任务中成功率从29%提升至56%,零样本迁移从15%到33%。
  • 轻量级设计兼容现有模型,适合需要精准控制的机器人应用。

离散动作标记为自回归视觉-语言-动作(VLA)策略提供了紧凑接口,但准确从离散代码恢复连续机器人动作仍具挑战。现有标记器通常将每个离散代码映射到固定的连续动作原型,忽略机器人当前本体感受状态。这一局限在操作任务中尤为明显,相同动作标记在不同关节配置、物体位姿和接触条件下需不同连续控制。为此,我们提出SA-VLA,一种状态感知的动作标记器,使动作解码依赖于机器人状态。研究了两种基于VQ的动作标记状态注入机制:状态与动作特征间的交叉注意力,以及轻量级状态适配器,用于预测动作级调制因子以实现状态条件下的动作调制与重构。适配器结构通过允许每个离散标记代表一族状态相关的连续动作,扩展了有限码本的有效支持,同时保持离散动作建模的高效性与兼容性。集成到基于LLM的VLA策略中,SA-VLA支持自回归与并行动作标记解码,对模型接口改动极小。在12个RoboTwin操作任务中,平均成功率从0.29提升至0.56;在三个真实世界任务的零样本仿真到现实迁移实验中,平均成功率从0.15提升至0.33。结果表明,状态条件动作解码是缓解离散VLA策略压缩差距的一种简单而有效机制。

原文摘要 · Abstract (English)

Discrete action tokenization provides a compact interface for autoregressive VLA policies, but accurately recovering continuous robot actions from discrete codes remains challenging. Existing tokenizers typically map each discrete code to a fixed continuous action prototype, ignoring the robot's current proprioceptive state. This limitation is particularly pronounced in manipulation, where the same action token may require different continuous controls under different joint configurations, object poses, and contact conditions. We therefore propose SA-VLA, a state-aware action tokenizer that conditions action decoding on robot state. We study two state-injection mechanisms for VQ-based action tokenization: cross-attention between state and action features, and a lightweight state adapter that predicts action-wise modulation factors for state-conditioned action modulation and reconstruction. The adapter formulation expands the effective support of a finite codebook by allowing each discrete token to represent a family of state-dependent continuous actions, while preserving the efficiency and compatibility of discrete action modeling. Integrated into an LLM-based VLA policy, SA-VLA supports both autoregressive and parallel action-token decoding with minimal changes to the model interface. On 12 RoboTwin manipulation tasks, SA-VLA improves the average success rate from 0.29 to 0.56 over the strongest tokenizer baseline. In zero-shot sim-to-real experiments on three real-world tasks, it further improves average success from 0.15 to 0.33 over the strongest tokenizer baseline. These results demonstrate that state-conditioned action decoding is a simple and effective mechanism for reducing the compression gap in discrete VLA policies.

机器人控制动作标记状态感知视觉语言动作

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。