深度与遮挡边界联合建模,提升两者精度与几何一致性。
Occlusion Boundary and Depth: Mutual Enhancement via Multi-Task Learning
- 设计跨注意力模块融合遮挡边界特征指导深度预测。
- 提出约束损失函数,确保深度与遮挡边界的几何一致。
- 在合成数据上训练可直接用于真实场景,无需微调。
遮挡边界估计(OBE)识别物体间遮挡及单个物体自遮挡形成的边界。该任务与单目深度估计(MDE)密切相关:遮挡边界为解决深度模糊提供关键几何线索,而深度信息也可反向优化遮挡判断。本文系统建模二者互益关系,提出MoDOT框架,联合估计深度与遮挡边界。其引入新的交叉注意力条带模块(CASM),利用中层遮挡边界特征增强深度预测;并设计遮挡-深度约束损失(OBDCL)以强制几何一致性。为支持研究,构建了大规模真实感数据集OB-Hypersim,包含精确深度与自遮挡处理的遮挡边界标注。在两个合成数据集及NYUD-v2上的实验表明,MoDOT显著优于单任务基线与多任务对比方法。且仅在合成数据训练的模型在真实场景中表现良好,无需微调,生成具有更锐利边界和更高几何保真度的深度图。结果验证了联合建模的显著优势。
原文摘要 · Abstract (English)
Occlusion Boundary Estimation (OBE) identifies boundaries arising from both inter-object occlusions and self-occlusion within individual objects. This task is closely related to Monocular Depth Estimation (MDE), which infers depth from a single image, as Occlusion Boundaries (OBs) provide critical geometric cues for resolving depth ambiguities, while depth can conversely refine occlusion reasoning. In this paper, we aim to systematically model and exploit this mutually beneficial relationship. To this end, we propose MoDOT, a novel framework for joint estimation of depth and OBs, which incorporates a new Cross-Attention Strip Module (CASM) to leverage mid-level OB features for depth prediction, and a novel OB-Depth Constraint Loss (OBDCL) to enforce geometric consistency. To facilitate this study, we contribute OB-Hypersim, a large-scale photorealistic dataset with precise depth and self-occlusion-handled OB annotations. Extensive experiments on two synthetic datasets and NYUD-v2 demonstrate that MoDOT achieves significantly better performance than single-task baselines and multi-task competitors. Furthermore, models trained solely on our synthetic data demonstrate strong generalization to real-world scenes without fine-tuning, producing depth maps with sharper boundaries and improved geometric fidelity. Collectively, these results underscore the significant benefits of jointly modeling OBs and depth. Code and resources are available at https://github.com/xul-ops/MoDOT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。