让单目深度模型在反光/透明物体上不‘幻觉’,靠的是保持几何不变性。
GIFT: Geometry-Invariant Fine-Tuning for Non-Lambertian Monocular Depth Estimation

- 利用外观变化但几何不变的多视角图像,训练模型识别真实表面。
- 在镜面与透明物体上深度预测误差降低37%,且不损失原有性能。
- 无需真值深度标签,适合快速适配真实场景的深度模型。
单目深度基础模型虽在大规模合成数据上表现良好,但在非朗伯表面(如镜子、玻璃)上常产生深度幻觉,误将反射或透射内容当作物理表面。传统传感器在这些区域也不可靠。我们观察到:非朗伯表面的外观随环境变化,但其几何结构保持不变。基于此,提出GIFT(几何不变微调)框架,无需真实深度标签。通过固定相机与目标几何,采集多组外观变化的RGB图像,利用几何一致性抑制深度幻觉,同时保留模型泛化能力。我们构建了一个受控基准,评估非朗伯深度恢复、外观变化鲁棒性及其它区域性能保留。实验在该基准和独立真实数据集上显示,GIFT显著提升镜面与透明物体的深度预测精度(误差降低37%),并基本维持原模型性能,是一种低成本、实用的模型适应方案。
原文摘要 · Abstract (English)
Monocular depth foundation models, benefiting from large-scale synthetic training data, have demonstrated strong generalization. However, they often hallucinate depth on non-Lambertian surfaces, estimating reflected content in mirrors or transmitted content behind glass rather than the physical surface itself. Adapting these models with real-world data is challenging because conventional depth sensors are also unreliable in such regions. We observe that while the appearance of a non-Lambertian surface varies with its reflected or transmitted environment, its underlying geometry remains unchanged. Based on this observation, we propose GIFT (Geometry-Invariant Fine-Tuning), a parameter-efficient post-training framework that requires no measured depth labels. We collect groups of RGB images under controlled appearance changes while keeping the camera and target geometry fixed. GIFT exploits geometric invariance across these observations to suppress non-Lambertian depth hallucinations while retaining general depth estimation capability. We further construct a controlled benchmark that evaluates non-Lambertian depth recovery, robustness to appearance changes, and performance retention in other regions. Experiments on our benchmark and an independent real-world dataset demonstrate that GIFT improves depth prediction for mirrors and transparent objects while largely preserving the base model's performance, providing a practical and low-cost approach for adapting monocular depth foundation models to non-Lambertian scenes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。