用单个语义标记压缩每帧视觉信息，提升长时序机器人决策性能。
One Token Per Frame: Reconsidering Visual Bandwidth in World Models for VLA Policy
- 每帧仅保留一个语义标记，通过自适应注意力池化实现视觉压缩
- 在MetaWorld等任务上成功率提升至61.3%，真实机械臂折叠布料任务达60.0%
- 统一流匹配目标，避免额外解码器，适合资源受限的机器人部署
视觉-语言-动作（VLA）模型越来越多依赖世界模型进行长时序规划，但如何在预训练VLA基础上参数化世界模块仍是一个开放问题。现有方法通常以高视觉带宽将每帧视觉流输入世界模型，并将滚动结果视为动作预测的副产品；在冻结主干网络的适配预算下，这导致每帧表示与潜在动作耦合均未被充分探索。我们提出OneWM-VLA，通过自适应注意力池化将每帧图像压缩为单一语义标记，统一生成潜在序列与动作轨迹，采用单一流匹配目标而非独立解码器连接。实验表明，在1471万LoRA参数、π₀（2B）主干网络下，OneWM-VLA在MetaWorld~MT50上平均成功率从47.9%提升至61.3%，在LIBERO-Long上达95.6%（π₀为85.2%），真实Piper机械臂上长时序折布任务成功率从20.0%提升至60.0%。
原文摘要 · Abstract (English)
Vision-language-action (VLA) models increasingly rely on auxiliary world modules to plan over long horizons, yet how such modules should be parameterized on top of a pretrained VLA remains an open design question. Existing world-model-augmented VLAs typically pass the per-frame visual stream into the world module at high visual bandwidth and treat its rollout as a side product of action prediction; under a constrained adaptation budget on a frozen backbone, this leaves both the per-frame representation and the latent action coupling under-examined. We introduce OneWM-VLA, which compresses each view into a single semantic token per frame through an Adaptive Attention Pooling, and produces the resulting latent stream and the action trajectory under a single flow-matching objective rather than connecting them through a separate decoder. Empirically, we find that per-frame visual bandwidth can be reduced to a single token without compromising long-horizon performance under our setup. Trained with 14.71M LoRA parameters on a $π_0$ (2B) backbone, OneWM-VLA improves the average success rate from 47.9% to 61.3% on MetaWorld~MT50, reaches 95.6% on LIBERO-Long (vs.85.2% for $π_0$), and reaches 60.0% on the long-horizon deformable task Fold Cloth on a real Piper arm (vs.20.0% for $π_0$).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。