arXiv:2604.07664cs.CVeess.IV2026-04中稿 · IEEE TMM

用扩散模型修复特征,提升单目深度估计精度

Monocular Depth Estimation From the Perspective of Feature Restoration: A Diffusion Enhanced Depth Restoration Approach

论文配图:Monocular Depth Estimation From the Perspective of Feature Restoration: A Diffusion Enhanced Depth Restoration Approach
图 1 · 摘自论文原文
  • 将深度估计视为特征恢复问题,利用扩散模型修复编码器特征
  • 在KITTI上相比基线RMSE降低4.09%~37.77%
  • 支持多视角辅助增强细节,适合追求高精度的3D视觉研究者

单目深度估计(MDE)是3D视觉中的基础任务。当前主流方法采用多层级特征处理的编码器-解码器架构,但其架构局限性及不同层级特征对预测精度的影响尚未充分评估。本文首次研究该问题,发现若能改进编码器特征,则现有框架仍有较大潜力。因此,提出从特征恢复视角重构深度估计:将预训练编码器特征视为假定真实特征的退化版本,该真实特征生成真实深度图。为此设计了基于可逆变换的间接扩散模块(InvT-IndDiffusion),通过满足双利普希茨条件的可逆解码器,在仅依赖最终稀疏深度图间接监督的情况下,有效缓解扩散迭代过程中的特征偏移。此外,提出一个即插即用的辅助视角低层特征增强模块(AV-LFE),在有辅助视角时增强局部细节。实验表明,所提方法在多个数据集上优于现有最优方法。特别地,在KITTI基准上,不同训练设置下相比基线,RMSE分别提升4.09%和37.77%。代码已公开于https://github.com/whitehb1/IID-RDepth。

原文摘要 · Abstract (English)

Monocular Depth Estimation (MDE) is a fundamental computer vision task with important applications in 3D vision. The current mainstream MDE methods employ an encoder-decoder architecture with multi-level/scale feature processing. However, the limitations of the current architecture and the effects of different-level features on the prediction accuracy are not evaluated. In this paper, we first investigate the above problem and show that there is still substantial potential in the current framework if encoder features can be improved. Therefore, we propose to formulate the depth estimation problem from the feature restoration perspective, by treating pretrained encoder features as degraded features of an assumed ground truth feature that yields the ground truth depth map. Then an Invertible Transform-enhanced Indirect Diffusion (InvT-IndDiffusion) module is developed for feature restoration. Due to the absence of direct supervision on feature, only indirect supervision from the final sparse depth map is used. During the iterative procedure of diffusion, this results in feature deviations among steps. The proposed InvT-IndDiffusion solves this problem by using an invertible transform-based decoder under the bi-Lipschitz condition. Finally, a plug-and-play Auxiliary Viewpoint-based Low-level Feature Enhancement module (AV-LFE) is developed to enhance local details with auxiliary viewpoint when available. Experiments demonstrate that the proposed method achieves better performance than the state-of-the-art methods on various datasets. Specifically on the KITTI benchmark, compared with the baseline, the performance is improved by 4.09% and 37.77% under different training settings in terms of RMSE. Code is available at https://github.com/whitehb1/IID-RDepth.

深度估计扩散模型特征恢复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。