用RGB图引导深度图超分辨率，提升细节精度
IGAF: Incremental Guided Attention Fusion for Depth Super-Resolution
- 分阶段注意力融合模块，精准结合RGB与低分辨率深度图
- 在NYUv2上实现×4、×8、×16超分的顶尖表现
- 无需微调即在多个数据集上超越现有方法，适合实际部署
精确的深度估计对机器人、导航和医学成像等领域至关重要。然而，传统深度传感器常生成低分辨率（LR）深度图，难以实现精细场景感知。为此，将低分辨率深度图超分辨率至高分辨率（HR）成为关键，需借助如RGB或灰度图像等高分辨率结构化输入进行引导。本文提出一种新的传感器融合方法——引导式深度超分辨率（GDSR），通过融合低分辨率深度图与高分辨率图像来生成高分辨率深度图。核心贡献是增量式引导注意力融合（IGAF）模块，能有效学习融合来自RGB图像和低分辨率深度图的特征，生成高精度高分辨率深度图。基于IGAF构建的模型在多个基准数据集上进行了评估。在NYUv2数据集上，对于×4、×8、×16的上采样倍数，均达到当前最优性能。同时，在Middlebury、Lu、RGB-D-D数据集的零样本设置下也全面超越所有基线模型。代码、环境与模型已公开于GitHub。
原文摘要 · Abstract (English)
Accurate depth estimation is crucial for many fields, including robotics, navigation, and medical imaging. However, conventional depth sensors often produce low-resolution (LR) depth maps, making detailed scene perception challenging. To address this, enhancing LR depth maps to high-resolution (HR) ones has become essential, guided by HR-structured inputs like RGB or grayscale images. We propose a novel sensor fusion methodology for guided depth super-resolution (GDSR), a technique that combines LR depth maps with HR images to estimate detailed HR depth maps. Our key contribution is the Incremental guided attention fusion (IGAF) module, which effectively learns to fuse features from RGB images and LR depth maps, producing accurate HR depth maps. Using IGAF, we build a robust super-resolution model and evaluate it on multiple benchmark datasets. Our model achieves state-of-the-art results compared to all baseline models on the NYU v2 dataset for $\times 4$, $\times 8$, and $\times 16$ upsampling. It also outperforms all baselines in a zero-shot setting on the Middlebury, Lu, and RGB-D-D datasets. Code, environments, and models are available on GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。