同时去雾与深度估计,用共享网络提升真实驾驶视频的清晰度和深度精度。
Depth-Centric Dehazing and Depth-Estimation from Real-World Hazy Driving Video
- 共享深度网络联动去雾与深度估计,利用前后帧一致性增强深度。
- 在真实雾霾场景中,去雾和深度估计性能均超越现有方法。
- 适合自动驾驶、智能车视觉系统等需要高精度感知的场景。
本文研究从单目雾霾视频中同步去除雾霾并估计深度的挑战性问题。这两项任务具有内在互补性:更精确的深度估计可通过大气散射模型(ASM)提升去雾效果,而更优的去雾结果则通过亮度一致性约束(BCC)促进深度估计。为此,我们提出一种新的以深度为中心的学习框架,将ASM与BCC约束融合。核心思想是,ASM与BCC均依赖于一个共享的深度估计网络:该网络利用相邻去雾帧增强深度估计(通过BCC),并利用优化后的深度线索更有效地进行去雾(通过ASM)。此外,我们引入非对齐的清晰视频及其估计深度,分别对去雾与深度网络进行独立正则化。具体设计两个判别器:$D_{MFIR}$ 增强去雾视频的高频细节,$D_{MDR}$ 减少低纹理区域的黑色空洞现象。大量实验表明,所提方法在真实雾霾场景下的视频去雾与深度估计任务中均优于当前最先进方法。
原文摘要 · Abstract (English)
In this paper, we study the challenging problem of simultaneously removing haze and estimating depth from real monocular hazy videos. These tasks are inherently complementary: enhanced depth estimation improves dehazing via the atmospheric scattering model (ASM), while superior dehazing contributes to more accurate depth estimation through the brightness consistency constraint (BCC). To tackle these intertwined tasks, we propose a novel depth-centric learning framework that integrates the ASM model with the BCC constraint. Our key idea is that both ASM and BCC rely on a shared depth estimation network. This network simultaneously exploits adjacent dehazed frames to enhance depth estimation via BCC and uses the refined depth cues to more effectively remove haze through ASM. Additionally, we leverage a non-aligned clear video and its estimated depth to independently regularize the dehazing and depth estimation networks. This is achieved by designing two discriminator networks: $D_{MFIR}$ enhances high-frequency details in dehazed videos, and $D_{MDR}$ reduces the occurrence of black holes in low-texture regions. Extensive experiments demonstrate that the proposed method outperforms current state-of-the-art techniques in both video dehazing and depth estimation tasks, especially in real-world hazy scenes. Project page: https://fanjunkai1.github.io/projectpage/DCL/index.html.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。