用量化深度预测提升视觉语言动作模型的空间理解能力
QDepth-VLA: Quantized Depth Prediction as Auxiliary Supervision for Vision-Language-Action Models
- 引入量化深度预测作为辅助任务，增强模型对3D结构的感知
- 在仿真和真实场景中均实现更强的空间推理与操控性能
- 适合需要精确空间理解的机器人操控研究者
空间感知与推理对于视觉-语言-动作（VLA）模型完成精细操作任务至关重要。然而，现有方法往往缺乏对精确控制所需关键3D结构的理解与推理能力。为此，我们提出QDepth-VLA，一种通过辅助深度预测任务增强VLA模型的通用框架。设计专用深度专家，预测由VQ-VAE编码器获得的深度图量化潜在标记，使模型学习具备深度感知的表征，捕捉关键几何线索。在仿真基准和真实世界任务上的实验结果表明，QDepth-VLA显著提升了空间推理能力，并在操控任务上达到具有竞争力的表现。
原文摘要 · Abstract (English)
Spatial perception and reasoning are crucial for Vision-Language-Action (VLA) models to accomplish fine-grained manipulation tasks. However, existing approaches often lack the ability to understand and reason over the essential 3D structures necessary for precise control. To address this limitation, we propose QDepth-VLA, a general framework that augments VLA models with an auxiliary depth prediction task. A dedicated depth expert is designed to predict quantized latent tokens of depth maps obtained from a VQ-VAE encoder, enabling the model to learn depth-aware representations that capture critical geometric cues. Experimental results on the simulation benchmarks and real-world tasks demonstrate that QDepth-VLA yields strong spatial reasoning and competitive performance on manipulation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。