用少量标注实现跨场景高精度深度补全,提升模型泛化能力。
PacGDC: Label-Efficient Generalizable Depth Completion with Projection Ambiguity and Consistency
- 利用2D到3D投影中的形状与位置模糊性,合成多样伪几何结构。
- 在零样本和少样本下,在多个基准上均超越现有方法。
- 适合需要低标注成本、强泛化性的自动驾驶与机器人应用。
通用深度补全可为未见环境生成稠密度量深度图,支持下游任务的鲁棒感知。但训练通常依赖大规模带度量标签的数据集,采集成本高。本文提出PacGDC,一种标签高效的通用深度补全技术,通过挖掘2D到3D投影中物体形状与位置的内在模糊性与一致性,为同一视觉场景合成大量伪几何结构。该过程通过操控对应深度图的场景尺度,显著扩展可用几何多样性。为此,我们设计新的数据合成流水线,使用多个深度基础模型作为尺度调节器,鲁棒生成具有不同场景尺度的伪深度标签,影响局部物体与全局布局,同时保证投影一致性以支持泛化。为进一步丰富几何形态,引入插值与重定位策略及无标签图像,拓展数据覆盖范围。大量实验表明,PacGDC在多个基准上展现出卓越泛化能力,无论在零样本或少样本设置下,均在多样化场景语义/尺度及深度稀疏性/模式上表现优异。
原文摘要 · Abstract (English)
Generalizable depth completion enables the acquisition of dense metric depth maps for unseen environments, offering robust perception capabilities for various downstream tasks. However, training such models typically requires large-scale datasets with metric depth labels, which are often labor-intensive to collect. This paper presents PacGDC, a label-efficient technique that enhances data diversity with minimal annotation effort for generalizable depth completion. PacGDC builds on novel insights into inherent ambiguities and consistencies in object shapes and positions during 2D-to-3D projection, allowing the synthesis of numerous pseudo geometries for the same visual scene. This process greatly broadens available geometries by manipulating scene scales of the corresponding depth maps. To leverage this property, we propose a new data synthesis pipeline that uses multiple depth foundation models as scale manipulators. These models robustly provide pseudo depth labels with varied scene scales, affecting both local objects and global layouts, while ensuring projection consistency that supports generalization. To further diversify geometries, we incorporate interpolation and relocation strategies, as well as unlabeled images, extending the data coverage beyond the individual use of foundation models. Extensive experiments show that PacGDC achieves remarkable generalizability across multiple benchmarks, excelling in diverse scene semantics/scales and depth sparsity/patterns under both zero-shot and few-shot settings. Code: https://github.com/Wang-xjtu/PacGDC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。