arXiv:2502.16421cs.CV2025-02被引 1

用渲染学习生成逼真可控的暴雨场景,提升自动驾驶感知模型测试效果

Learning from Rendering: Realistic and Controllable Extreme Rainy Image Synthesis for Autonomous Driving Simulation

  • 结合渲染真实感与学习方法可控性,实现光照变化下的暴雨图像生成
  • 在合成数据上使语义分割模型mIoU提升5%~8%,真实暴雨场景下表现显著增强
  • 适合需要极端天气测试的自动驾驶仿真研究者使用

自动驾驶模拟器为评估或提升视觉感知模型提供了高效低成本的替代方案。然而,评估可靠性依赖于场景的多样性和真实性。极端天气(尤其是暴雨)在真实环境中稀少且采集成本高。现有雨景生成方法常因光照控制差、真实感不足而影响模型评估效果。为此,我们提出一种基于渲染学习的雨景生成方法,融合了渲染方法的真实感与学习方法的可控性。为验证其在语义分割任务中的有效性,需构建连续标注的极端雨景图像集。通过将该生成器与CARLA驾驶模拟器结合,我们开发了CARLARain——一个可生成复杂光照条件下成对雨天/晴天图像及标签的极端雨景模拟系统。定性与定量实验表明,使用CARLARain训练的模型在合成数据上的语义分割准确率(mIoU)提升5%~8%,并在真实极端雨天场景中表现出显著增强性能。代码与数据集已开源。

原文摘要 · Abstract (English)

Autonomous driving simulators provide an effective and low-cost alternative for evaluating or enhancing visual perception models. However, the reliability of evaluation depends on the diversity and realism of the generated scenes. Extreme weather conditions, particularly extreme rainfalls, are rare and costly to capture in real-world settings. While simulated environments can help address this limitation, existing rainy image synthesizers often suffer from poor controllability over illumination and limited realism, which significantly undermines the effectiveness of the model evaluation. To that end, we propose a learning-from-rendering rainy image synthesizer, which combines the benefits of the realism of rendering-based methods and the controllability of learning-based methods. To validate the effectiveness of our extreme rainy image synthesizer on semantic segmentation task, we require a continuous set of well-labeled extreme rainy images. By integrating the proposed synthesizer with the CARLA driving simulator, we develop CARLARain an extreme rainy street scene simulator which can obtain paired rainy-clean images and labels under complex illumination conditions. Qualitative and quantitative experiments validate that CARLARain can effectively improve the accuracy of semantic segmentation models in extreme rainy scenes, with the models' accuracy (mIoU) improved by 5% - 8% on the synthetic dataset and significantly enhanced in real extreme rainy scenarios under complex illuminations. Our source code and datasets are available at https://github.com/kb824999404/CARLARain/.

自动驾驶图像合成极端天气语义分割

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。