通过合成数据桥接,实现恶劣天气下光流的渐进式精准迁移。
Adverse Weather Optical Flow: Cumulative Homogeneous-Heterogeneous Adaptation
- 分步融合静态/动态天气特征,构建同质-异质自适应框架。
- 在RealWeatherFlow数据集上,mACE降低至1.72,优于现有方法。
- 适合需要高鲁棒性光流估计的自动驾驶与视觉系统研发者。
光流在晴朗场景中已取得显著进展,但在恶劣天气下因亮度恒定性和梯度连续性假设失效而性能下降。现有方法多采用单阶段域适应,从干净域向真实退化域迁移运动知识,但因干净域与真实退化域间存在巨大差距,且退化域内静态(如雾)与动态(如雨)天气影响不同,导致效果不佳。为此,本文探索以合成退化域为中间桥梁,提出累积同质-异质适应框架。关键洞察:静态天气具有深度关联的同质特性,不改变场景内在运动;动态天气引入异质特性,导致变形误差边界显著差异。对于合成到真实域的迁移,发现代价体相关性在合成与真实退化域间具有相似统计直方图,有利于整体对齐同质相关分布,实现知识蒸馏。该框架可逐步、显式地将干净场景知识迁移至真实恶劣天气。此外,我们构建了带人工标注光流标签的真实恶劣天气数据集,并通过大量实验验证了方法优势。
原文摘要 · Abstract (English)
Optical flow has made great progress in clean scenes, while suffers degradation under adverse weather due to the violation of the brightness constancy and gradient continuity assumptions of optical flow. Typically, existing methods mainly adopt domain adaptation to transfer motion knowledge from clean to degraded domain through one-stage adaptation. However, this direct adaptation is ineffective, since there exists a large gap due to adverse weather and scene style between clean and real degraded domains. Moreover, even within the degraded domain itself, static weather (e.g., fog) and dynamic weather (e.g., rain) have different impacts on optical flow. To address above issues, we explore synthetic degraded domain as an intermediate bridge between clean and real degraded domains, and propose a cumulative homogeneous-heterogeneous adaptation framework for real adverse weather optical flow. Specifically, for clean-degraded transfer, our key insight is that static weather possesses the depth-association homogeneous feature which does not change the intrinsic motion of the scene, while dynamic weather additionally introduces the heterogeneous feature which results in a significant boundary discrepancy in warp errors between clean and degraded domains. For synthetic-real transfer, we figure out that cost volume correlation shares a similar statistical histogram between synthetic and real degraded domains, benefiting to holistically aligning the homogeneous correlation distribution for synthetic-real knowledge distillation. Under this unified framework, the proposed method can progressively and explicitly transfer knowledge from clean scenes to real adverse weather. In addition, we further collect a real adverse weather dataset with manually annotated optical flow labels and perform extensive experiments to verify the superiority of the proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。