arXiv:2512.04734cs.CV2025-12

用实例掩码引导深度补全,提升边界和遮挡处精度

MT-Depth: Multi-task Instance feature analysis for the Depth Completion

  • 引入实例掩码作为空间先验,通过跨注意力融合指导深度预测
  • 在Virtual KITTI 2上实现更低的RMSE,优于仅用U-Net或语义引导的方法
  • 适合自动驾驶、机器人等需精细深度感知的场景

深度补全在3D感知系统中至关重要,尤其在稀疏深度数据需稠密化时,如自动驾驶、机器人和增强现实。现有方法多依赖语义分割引导深度补全,但忽略了物体级理解的优势。本文提出一种实例感知的深度补全框架,显式将二值实例掩码作为空间先验,以优化深度预测。模型包含四个组件:冻结的YOLO V11实例分割分支、基于U-Net的深度补全主干、跨注意力融合模块及注意力引导的预测头。实例分割分支生成图像级前景掩码,通过跨注意力机制引导深度分支,使网络在细化过程中聚焦于物体中心区域。在Virtual KITTI 2数据集上验证表明,该方法相比仅用U-Net的基线和先前语义引导方法,取得了更低的均方根误差(RMSE),同时保持了具有竞争力的平均绝对误差(MAE)。定性和定量结果均显示,所提模型有效提升了物体边界、遮挡区及细结构处的深度精度。研究结果表明,引入实例级线索为提升深度补全性能提供了有前景的方向,且无需密集语义标签。

原文摘要 · Abstract (English)

Depth completion plays a vital role in 3D perception systems, especially in scenarios where sparse depth data must be densified for tasks such as autonomous driving, robotics, and augmented reality. While many existing approaches rely on semantic segmentation to guide depth completion, they often overlook the benefits of object-level understanding. In this work, we introduce an instance-aware depth completion framework that explicitly integrates binary instance masks as spatial priors to refine depth predictions. Our model combines four main components: a frozen YOLO V11 instance segmentation branch, a U-Net-based depth completion backbone, a cross-attention fusion module, and an attention-guided prediction head. The instance segmentation branch generates per-image foreground masks that guide the depth branch via cross-attention, allowing the network to focus on object-centric regions during refinement. We validate our method on the Virtual KITTI 2 dataset, showing that it achieves lower Root Mean Squared Error (RMSE) compared to both a U-Net-only baseline and previous semantic-guided methods, while maintaining competitive Mean Absolute Error (MAE). Qualitative and quantitative results demonstrate that the proposed model effectively enhances depth accuracy near object boundaries, occlusions, and thin structures. Our findings suggest that incorporating instance-aware cues offers a promising direction for improving depth completion without relying on dense semantic labels.

深度补全实例分割跨注意力3D感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。