将视觉语言模型的冗余信息压缩,提升机器人控制的泛化能力。
QuoVLA: Quotient Space for Vision-Language-Action Models
- 通过商空间理论压缩预训练模型特征,保留控制所需信息。
- 在多类分布偏移下,泛化性能显著优于现有方法。
- 适合关注机器人视觉-语言-动作统一建模的研究者。
视觉-语言-动作(VLA)模型通常通过将视觉观测和语言指令映射为连续动作来适配机器人控制。现有方法普遍持‘动作信息不足’观点,认为预训练视觉-语言模型(VLM)特征中缺乏可直接使用的动作信息,或应屏蔽动作学习信号。本文提出‘商空间用于VLA’理论,指出预训练VLM特征并非动作不足,而是过完备:其区分了导致相同最优动作行为的提示级差异。为此,我们提出QuoVLA,一种基于商空间的VLA框架,通过量化模块与双分支结构结合相对时序复杂度正则化,将预训练VLM特征压缩为动作充分表示,保留动作相关信息并消除提示级冗余。大量实验表明,QuoVLA在多个基准上表现优异,尤其在视觉、语言及环境分布偏移下的泛化能力显著提升。代码将公开。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models commonly adapt pretrained Vision-Language Models (VLMs) to robot control by mapping visual observations and language instructions to continuous actions. Existing approaches typically take an action-insufficiency view, assuming that pretrained VLM latents either lack directly usable action information or should be shielded from action-learning signals. Against this view, our \textit{Quotient Theory for VLA} shows that pretrained VLM latents are not action-insufficient but action-sufficient: they already contain the information needed for control, yet remain overcomplete by distinguishing prompt-level variations that induce the same optimal action behavior. To operationalize this theory, we propose QuoVLA, a quotient-space framework for VLA that compresses pretrained VLM latents into action-sufficient representations. Specifically, QuoVLA instantiates this principle with a quantization module and a dual-branch design with relative temporal-complexity regularization, preserving action-relevant information while removing prompt-level redundancy. Extensive experiments across multiple benchmarks demonstrate that QuoVLA achieves strong performance, with particularly notable improvements in generalization under visual, linguistic, and environmental distribution shifts. Our code will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。