arXiv:2604.03191cs.ROcs.CV2026-04被引 3

离散动作编码限制了视觉-语言-动作模型的扩展,导致升级视觉模型也无效。

The Compression Gap: Why Discrete Tokenization Limits Vision-Language-Action Model Scaling

  • 用信息论解释:当动作离散化时,代码本成最大瓶颈,阻碍性能提升。
  • 实验显示,升级视觉编码器使连续动作模型性能提升超21个百分点。
  • 放松代码本容量可恢复对视觉编码器的敏感性,验证瓶颈理论。

通过升级视觉编码器来扩展视觉-语言-动作(VLA)模型,预期能提升下游操作性能——这在视觉-语言建模中已得到证实。然而我们发现,当动作以离散令牌表示时,这一预期失效。我们通过信息论原理‘压缩间隙’解释其原因:在任何视觉运动流程中,性能扩展受限于最紧的信息瓶颈。当动作为连续形式(如扩散策略),视觉编码器是主要约束,升级它可直接提升性能;而当动作通过固定容量的代码本离散化(如OAT),代码本成为瓶颈,编码器改进无法传递过去——无论上游表征多么丰富。我们在LIBERO基准上通过三重证据验证该原则:因子实验表明,编码器升级使扩散策略性能提升超21个百分点,而OAT收益随模型规模显著衰减;四个编码器的质量梯度实验确认,扩散策略性能随编码器质量单调上升,而OAT保持平坦;代码本容量实验显示,放宽代码本容量可部分恢复编码器敏感性,为瓶颈假说提供因果证据。研究揭示,物理AI的扩展需识别流程中的信息瓶颈,而非盲目增加模型或数据规模。

原文摘要 · Abstract (English)

Scaling Vision-Language-Action (VLA) models by upgrading the vision encoder is expected to improve downstream manipulation performance--as it does in vision-language modeling. We show that this expectation fails when actions are represented as discrete tokens, and explain why through an information-theoretic principle we call the Compression Gap: in any visuomotor pipeline, scaling behavior is governed by the location of the tightest information bottleneck. When actions are continuous (e.g., Diffusion Policy), the vision encoder is the binding constraint, and upgrading it directly improves performance. When actions are discretized through a fixed-capacity codebook (e.g., OAT), the codebook becomes the binding constraint, and encoder improvements cannot propagate past it--regardless of how rich the upstream representation is. We validate this principle on the LIBERO benchmark with three lines of evidence: a factorial experiment showing that encoder upgrades improve Diffusion Policy by over 21 percentage points while OAT gains are substantially attenuated across model scales; an encoder quality gradient across four encoders confirming that Diffusion Policy tracks encoder quality monotonically while OAT remains flat; and a codebook size experiment demonstrating that relaxing codebook capacity partially recovers encoder sensitivity, providing causal evidence for the bottleneck hypothesis. Our findings reveal that scaling in Physical AI requires identifying where information bottlenecks lie in the pipeline, rather than uniformly increasing model or data size.

视觉-语言动作规划信息瓶颈模型扩展

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。