用检索邻居提升小众场景的单目深度估计精度
RAD: Retrieval-Augmented Monocular Metric Depth Estimation for Underrepresented Classes
- 通过不确定度筛选低信心区域，检索相似内容的RGB-D样本
- 双流网络融合+匹配交叉注意力，仅在可靠对应点传几何信息
- 在小众类别上误差降低超7%，适合复杂场景下弱数据任务
单目度量深度估计对物理智能系统至关重要，但在复杂场景中对少数类别的准确估计仍是难题。为此，我们提出RAD框架，通过检索邻近样本作为结构几何代理，模拟多视角立体效果。方法首先利用不确定性感知检索机制识别输入中低置信区域，并获取语义相似的RGB-D上下文样本；随后通过双流网络处理输入与检索样本，并采用匹配交叉注意力模块进行融合，仅在可靠点对应处传递几何信息。在NYU Depth v2、KITTI和Cityscapes上的评估表明，RAD在少数类别上显著优于现有基线，相对绝对误差分别降低29.2%（NYU Depth v2）、13.3%（KITTI）和7.2%（Cityscapes），同时在标准域基准上保持竞争力。
原文摘要 · Abstract (English)
Monocular Metric Depth Estimation (MMDE) is essential for physically intelligent systems, yet accurate depth estimation for underrepresented classes in complex scenes remains a persistent challenge. To address this, we propose RAD, a retrieval-augmented framework that approximates the benefits of multi-view stereo by utilizing retrieved neighbors as structural geometric proxies. Our method first employs an uncertainty-aware retrieval mechanism to identify low-confidence regions in the input and retrieve RGB-D context samples containing semantically similar content. We then process both the input and retrieved context via a dual-stream network and fuse them using a matched cross-attention module, which transfers geometric information only at reliable point correspondences. Evaluations on NYU Depth v2, KITTI, and Cityscapes demonstrate that RAD significantly outperforms state-of-the-art baselines on underrepresented classes, reducing relative absolute error by 29.2% on NYU Depth v2, 13.3% on KITTI, and 7.2% on Cityscapes, while maintaining competitive performance on standard in-domain benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。