通过几何约束注意力提升多视角深度估计的泛化能力
CrossDepth: Geometry-Constrained Attention for Generalizable Multi-View Surround Depth Estimation
- 用相机射线嵌入建模视角差异，增强单目线索一致性
- 跨图像注意力仅在几何合理区域扩展上下文，提升深度准确性
- 自监督训练，在真实数据集上优于现有方法
可靠的周围环境三维理解是自动驾驶的核心需求。多视角环视摄像头系统虽覆盖范围广，但相邻图像重叠区域极少，多数像素需依赖单目外观线索推断深度。这些线索在不同图像中表现各异，易导致深度模型产生不一致解读。本文针对两大跨图像不一致来源：相机内参差异与单图感受野有限。通过为每个像素引入相机感知的射线嵌入，使网络能适应单目线索的相机依赖性变化；并通过基于标定后硬件布局的几何可解释区域，对跨图像注意力进行约束，扩展每像素的上下文信息。模型采用基于光度一致性全自监督训练。在DDAD和nuScenes数据集上的评估表明，该方法在域内与跨域场景下均显著提升深度精度与跨图像一致性，优于当前最优自监督方法。代码已公开。
原文摘要 · Abstract (English)
Reliable 3D understanding of the surrounding environment is a core requirement for autonomous driving. Multi-view surround camera rigs provide broad scene coverage, but the spatially adjacent images typically overlap only minimally. Consequently, the depth of most pixels must be inferred from monocular appearance cues. These cues can appear differently across images and may therefore be interpreted differently by the depth estimation model. We target two main sources of cross-image inconsistency: differences in camera intrinsics and the limited receptive field of each image. We address the former by conditioning the features on per-pixel camera-aware ray embeddings, enabling the network to account for camera-dependent variations in monocular cues. We address the latter by extending each pixel's context beyond its own image through cross-image attention constrained to geometrically plausible regions, derived from the calibrated rig setup. The model is trained in a fully self-supervised manner based on photometric consistency. Evaluations on DDAD and nuScenes show improved overall depth accuracy and cross-image depth consistency over state-of-the-art self-supervised methods under in-domain and cross-domain evaluation. Code is available at https://abualhanud.github.io/CrossDepthPage/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。