轻量级模型通过多视角图像推断深度,提升机器人操作的空间感知能力。
Evo-Depth: A Lightweight Depth-Enhanced Vision-Language-Action Model

- 从多视角图像中隐式提取深度特征,不依赖额外传感器。
- 参数仅0.9B,在4个仿真基准上表现最优,真实场景成功率最高。
- 模型最小、内存占用低、推理频率高,适合部署在资源受限设备。
视觉-语言-动作模型在机器人操作中展现出巨大潜力,但常因缺乏深度信息而难以实现精准空间理解。现有方法或依赖深度图、点云等3D输入,增加系统复杂性与传感器成本;或使用大型几何基础模型进行隐式建模,导致训练和部署开销高。为此,我们提出Evo-Depth,一种轻量级深度增强型视觉-语言-动作框架。该框架通过轻量级隐式深度编码模块,从多视角RGB图像中提取紧凑深度特征,并通过空间增强模块以深度感知方式融合至视觉-语言表征中,实现高效的空间语义增强。进一步引入渐进对齐训练策略,使深度增强表征与下游动作学习有效对齐。模型仅含0.9B参数,在四个仿真基准上均取得领先性能。真实世界实验显示,其平均成功率最高,同时模型尺寸最小、GPU内存占用最低、推理频率最高,显著优于对比方法。
原文摘要 · Abstract (English)
Vision-Language-Action models have emerged as a promising paradigm for robotic manipulation by unifying perception, language grounding, and action generation. However, they often struggle in scenarios requiring precise spatial understanding, as current VLA models primarily rely on 2D visual representations that lack depth information and detailed spatial relationships. While recent approaches incorporate explicit 3D inputs such as depth maps or point clouds to address this issue, they often increase system complexity, require additional sensors, and remain vulnerable to sensing noise and reconstruction errors. Another line of work explores implicit 3D-aware spatial modeling directly from RGB observations without extra sensors, but it often relies on large geometry foundation models, resulting in higher training and deployment costs. To address these challenges, we propose Evo-Depth, a lightweight depth-enhanced VLA framework that enhances spatially grounded manipulation without relying on additional sensing hardware or compromising deployment efficiency. Evo-Depth employs a lightweight Implicit Depth Encoding Module to extract compact depth features from multi-view RGB images. These features are incorporated into vision-language representations through a Spatial Enhancement Module via depth-aware modulation, enabling efficient spatial-semantic enhancement. A Progressive Alignment Training strategy is further introduced to align the resulting depth-enhanced representations with downstream action learning. With only 0.9B parameters, Evo-Depth achieves superior performance across four simulation benchmarks. In real-world experiments, Evo-Depth attains the highest average success rate while also exhibiting the smallest model size, lowest GPU memory usage, and highest inference frequency among compared methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。