arXiv:2411.14865eess.IVcs.CV2024-11被引 4

首个针对光流估计的抗损坏鲁棒性基准,揭示模型在常见干扰下的表现差异。

Benchmarking the Robustness of Optical Flow Estimation to Corruptions

  • 设计7种时序噪声+17种图像噪声,构建双数据集基准
  • 发现局部信息丢失比视觉失真更影响光流精度
  • 适合自动驾驶、视频处理等领域模型可靠性研究

光流估计广泛应用于自动驾驶与视频编辑。尽管现有模型在多个基准上表现优异,但其鲁棒性鲜有研究。针对光流特有的时间特性,本文设计了7种专门用于评估光流模型鲁棒性的时序噪声,结合17种经典单图噪声(含先进PSF模糊模拟),建立首个光流鲁棒性基准KITTI-FC与GoPro-FC,包含域外(OOD)与域内(ID)两种设置。提出腐蚀鲁棒性误差(CRE)、腐蚀鲁棒性误差比(CREr)和相对腐蚀鲁棒性误差(RCRE)三项指标量化鲁棒性。对15种方法的29个模型变体进行评估,发现:1)模型绝对鲁棒性高度依赖于原始性能;2)削弱局部信息的干扰比降低视觉效果的更严重。论文还为模型设计与应用提供建议。代码与基准已开源。

原文摘要 · Abstract (English)

Optical flow estimation is extensively used in autonomous driving and video editing. While existing models demonstrate state-of-the-art performance across various benchmarks, the robustness of these methods has been infrequently investigated. Despite some research focusing on the robustness of optical flow models against adversarial attacks, there has been a lack of studies investigating their robustness to common corruptions. Taking into account the unique temporal characteristics of optical flow, we introduce 7 temporal corruptions specifically designed for benchmarking the robustness of optical flow models, in addition to 17 classical single-image corruptions, in which advanced PSF Blur simulation method is performed. Two robustness benchmarks, KITTI-FC and GoPro-FC, are subsequently established as the first corruption robustness benchmark for optical flow estimation, with Out-Of-Domain (OOD) and In-Domain (ID) settings to facilitate comprehensive studies. Robustness metrics, Corruption Robustness Error (CRE), Corruption Robustness Error ratio (CREr), and Relative Corruption Robustness Error (RCRE) are further introduced to quantify the optical flow estimation robustness. 29 model variants from 15 optical flow methods are evaluated, yielding 10 intriguing observations, such as 1) the absolute robustness of the model is heavily dependent on the estimation performance; 2) the corruptions that diminish local information are more serious than that reduce visual effects. We also give suggestions for the design and application of optical flow models. We anticipate that our benchmark will serve as a foundational resource for advancing research in robust optical flow estimation. The benchmarks and source code will be released at https://github.com/ZhonghuaYi/optical_flow_robustness_benchmark.

光流估计鲁棒性自动驾驶基准测试

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。