利用时空先验提升夜间单目深度估计,解决低光与动态模糊问题。
DASP: Self-supervised Nighttime Monocular Depth Estimation with Domain Adaptation of Spatiotemporal Priors
- 设计对抗分支提取白天的时空先验特征,增强夜间纹理缺失区域重建能力。
- 提出3D一致性投影损失,在共享3D空间中优化结构一致性和先验对齐。
- 在Oxford RobotCar和nuScenes数据集上达到顶尖性能,适合自动驾驶场景使用。
自监督单目深度估计在白天已取得显著进展,但在夜间因光照不足、可见度低而性能下降,如光线不足导致纹理缺失,运动物体引发模糊区域。为此,本文提出自监督框架DASP,利用时空先验进行夜间深度估计。DASP包含对抗分支与自监督分支:对抗分支中,设计含四个时空先验学习模块(SPLB)的判别器,SPLB融合空间时间学习模块(STLM)与轴向空间学习模块(ASLM),前者通过正交差分捕捉时间轴上的运动变化,后者采用局部非对称卷积结合全局轴向注意力捕获多尺度结构信息;自监督分支提出3D一致性投影损失,将目标帧与源帧双向投影至共享3D空间,计算投影后帧间3D差异作为损失,以优化3D结构一致性与白天先验对齐。在Oxford RobotCar和nuScenes数据集上的大量实验表明,本方法在夜间深度估计上达到当前最优性能,消融实验验证了各组件有效性。
原文摘要 · Abstract (English)
Self-supervised monocular depth estimation has achieved notable success under daytime conditions. However, its performance deteriorates markedly at night due to low visibility and varying illumination, e.g., insufficient light causes textureless areas, and moving objects bring blurry regions. To this end, we propose a self-supervised framework named DASP that leverages spatiotemporal priors for nighttime depth estimation. Specifically, DASP consists of an adversarial branch for extracting spatiotemporal priors and a self-supervised branch for learning. In the adversarial branch, we first design an adversarial network where the discriminator is composed of four devised spatiotemporal priors learning blocks (SPLB) to exploit the daytime priors. In particular, the SPLB contains a spatial-based temporal learning module (STLM) that uses orthogonal differencing to extract motion-related variations along the time axis and an axial spatial learning module (ASLM) that adopts local asymmetric convolutions with global axial attention to capture the multiscale structural information. By combining STLM and ASLM, our model can acquire sufficient spatiotemporal features to restore textureless areas and estimate the blurry regions caused by dynamic objects. In the self-supervised branch, we propose a 3D consistency projection loss to bilaterally project the target frame and source frame into a shared 3D space, and calculate the 3D discrepancy between the two projected frames as a loss to optimize the 3D structural consistency and daytime priors. Extensive experiments on the Oxford RobotCar and nuScenes datasets demonstrate that our approach achieves state-of-the-art performance for nighttime depth estimation. Ablation studies further validate the effectiveness of each component.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。