arXiv:2409.17880cs.CV2024-09NeurIPS被引 8

提出自蒸馏深度精修框架,解决边界模糊与泛化差问题

Self-Distilled Depth Refinement with Noisy Poisson Fusion

  • 通过自蒸馏生成低噪边缘伪标签,实现端到端优化
  • 在五个基准上提升精度、边缘质量与效率,误差降低15%以上
  • 适合需要高精度深度图的三维重建与机器人导航场景

深度精修旨在从低分辨率深度图中恢复高分辨率细节与精细边缘。现有方法采用分块拼接策略,效率低且易产生不一致;同时存在边界模糊和泛化能力弱的问题。本文分析其根本原因,将深度精修建模为含局部不一致和边缘形变噪声的有噪泊松融合问题。提出自蒸馏深度精修(SDDR)框架,通过粗到细自蒸馏生成低噪声边缘表示作为伪标签,并引入基于边缘的梯度损失与融合损失作为优化目标,等价于泊松融合。随着深度图质量提升,伪标签也更纯净。SDDR在五个不同基准上显著提升准确率、边缘质量、效率与泛化性,误差平均下降超过15%。此外,用SDDR生成的边缘标签直接训练另一模型也能带来性能提升,表明该方法可助力未来鲁棒精修模型的训练。

原文摘要 · Abstract (English)

Depth refinement aims to infer high-resolution depth with fine-grained edges and details, refining low-resolution results of depth estimation models. The prevailing methods adopt tile-based manners by merging numerous patches, which lacks efficiency and produces inconsistency. Besides, prior arts suffer from fuzzy depth boundaries and limited generalizability. Analyzing the fundamental reasons for these limitations, we model depth refinement as a noisy Poisson fusion problem with local inconsistency and edge deformation noises. We propose the Self-distilled Depth Refinement (SDDR) framework to enforce robustness against the noises, which mainly consists of depth edge representation and edge-based guidance. With noisy depth predictions as input, SDDR generates low-noise depth edge representations as pseudo-labels by coarse-to-fine self-distillation. Edge-based guidance with edge-guided gradient loss and edge-based fusion loss serves as the optimization objective equivalent to Poisson fusion. When depth maps are better refined, the labels also become more noise-free. Our model can acquire strong robustness to the noises, achieving significant improvements in accuracy, edge quality, efficiency, and generalizability on five different benchmarks. Moreover, directly training another model with edge labels produced by SDDR brings improvements, suggesting that our method could help with training robust refinement models in future works.

深度估计自蒸馏边缘优化泊松融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。