提出自适应融合的深度估计模型,提升真实场景下的鲁棒性。
DepthMamba with Adaptive Fusion
- 双分支结构融合单视角与多视角估计结果
- 在KITTI和DDAD上优于现有方法,动态物体等场景表现佳
- 引入Mamba提取特征并自适应选择可靠输出
多视角深度估计在多个基准上取得了优异性能。然而,当前大多数系统依赖理想相机位姿,这在自动驾驶等真实场景中难以获得。本文提出一个新基准,评估在不同噪声位姿下的深度估计鲁棒性。令人意外的是,现有单视角或融合方法在位姿噪声下会失效。为此,我们设计了一种双分支网络架构,融合单视角与多视角分支的深度估计结果。具体地,采用Mamba作为特征提取主干,并提出基于注意力的自适应融合机制,动态选择更可靠的估计结果。该方法在包含动态物体、无纹理区域等挑战场景中表现良好。消融实验验证了主干与融合模块的有效性,而基于KITTI和DDAD的评测表明,所提方法达到领先水平。
原文摘要 · Abstract (English)
Multi-view depth estimation has achieved impressive performance over various benchmarks. However, almost all current multi-view systems rely on given ideal camera poses, which are unavailable in many real-world scenarios, such as autonomous driving. In this work, we propose a new robustness benchmark to evaluate the depth estimation system under various noisy pose settings. Surprisingly, we find current multi-view depth estimation methods or single-view and multi-view fusion methods will fail when given noisy pose settings. To tackle this challenge, we propose a two-branch network architecture which fuses the depth estimation results of single-view and multi-view branch. In specific, we introduced mamba to serve as feature extraction backbone and propose an attention-based fusion methods which adaptively select the most robust estimation results between the two branches. Thus, the proposed method can perform well on some challenging scenes including dynamic objects, texture-less regions, etc. Ablation studies prove the effectiveness of the backbone and fusion method, while evaluation experiments on challenging benchmarks (KITTI and DDAD) show that the proposed method achieves a competitive performance compared to the state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。