arXiv:2605.10485cs.RO2026-05被引 1

让视觉模型具备空间感知能力,提升机器人操作精度。

VEGA: Visual Encoder Grounding Alignment for Spatially-Aware Vision-Language-Action Models

论文配图:VEGA: Visual Encoder Grounding Alignment for Spatially-Aware Vision-Language-Action Models
图 1 · 摘自论文原文
  • 在视觉编码器输出层对齐3D感知特征,避免语义混淆。
  • 在仿真和真实任务中均超越现有方法,性能更优。
  • 轻量级设计,推理无额外开销,适合实际部署。

精准的空间推理对机器人操作至关重要,但当前视觉-语言-动作(VLA)模型的视觉主干多基于2D图像预训练,缺乏显式的3D几何监督,导致表征缺乏准确的空间意识。现有隐式空间对齐方法通过与3D感知基础模型的特征对齐来部分解决此问题,但依赖经验性层搜索,并在大语言模型级别的视觉标记上进行对齐,此时空间结构已与语言语义纠缠,限制了泛化性和几何可解释性。本文提出VEGA(Visual Encoder Grounding Alignment),一种简单有效的框架,直接将VLA模型的视觉编码器输出与经过多视角一致3D高斯泼溅监督微调的DINOv2-FiT3D模型的空间感知特征对齐。通过在视觉编码器输出层进行对齐,确保空间意识在语言语义融合前被锚定,提供更可解释、更合理的对齐目标。对齐采用轻量级投影器,结合余弦相似度损失与标准动作预测目标联合训练,并在推理时丢弃,不引入额外计算开销。大量仿真基准和真实世界操作任务实验表明,VEGA持续优于现有隐式空间对齐基线,成为隐式空间对齐方法中的新最优水平。

原文摘要 · Abstract (English)

Precise spatial reasoning is fundamental to robotic manipulation, yet the visual backbones of current vision-language-action (VLA) models are predominantly pretrained on 2D image data without explicit 3D geometric supervision, resulting in representations that lack accurate spatial awareness. Existing implicit spatial grounding methods partially address this by aligning VLA features with those of 3D-aware foundation models, but they rely on empirical layer search and perform alignment on LLM-level visual tokens where spatial structure has already been entangled with linguistic semantics, limiting both generalizability and geometric interpretability. We propose VEGA (Visual Encoder Grounding Alignment), a simple yet effective framework that directly aligns the output of the VLA's visual encoder with spatially-aware features from DINOv2-FiT3D, a DINOv2 model fine-tuned with multi-view consistent 3D Gaussian Splatting supervision. By performing alignment at the visual encoder output level, VEGA grounds spatial awareness before any linguistic entanglement occurs, offering a more interpretable and principled alignment target. The alignment is implemented via a lightweight projector trained with a cosine similarity loss alongside the standard action prediction objective, and is discarded at inference time, introducing no additional computational overhead. Extensive experiments on simulation benchmark and real-world manipulation tasks demonstrate that VEGA consistently outperforms existing implicit spatial grounding baselines, establishing a new state-of-the-art among implicit spatial grounding methods for VLA models.

空间感知视觉模型机器人操作对齐方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。