用游戏引擎生成配对驾驶图像,分离光照变化影响
PairedGTA: Generating Driving Datasets for Controlled Photometric Shift Analysis

- 通过游戏引擎控制光照天气,保持场景几何与物体位置一致
- 生成的图像对可精准分析光照变化对分割模型的影响
- 适合研究视觉模型在恶劣天气下的鲁棒性
评估自动驾驶视觉感知系统性能对确保其在多样环境下的可靠运行至关重要。理想情况下,公平比较不同恶劣条件下的表现需要完全配对的同一场景在不同天气或光照下的图像,从而独立分析光度变化的影响,而不受几何或语义变化干扰。然而,真实数据集通常无法提供此类图像,因相机位姿、交通状况和动态物体(车辆、行人等)位置随时间变化,仅能获得粗略配对的数据。为此,本文提出一种基于高保真游戏引擎的图像生成框架,通过与GTA游戏引擎通信的软件API,修改光照与天气条件,同时保持场景几何结构、相机姿态及动态物体的身份与位置不变。在每个采样位置,程序化生成动态实体,并在多种恶劣条件下渲染像素对齐图像。该生成框架在驾驶场景中的优势通过语义分割模型的系统性分析得以验证,其性能下降可更直接归因于光度变化,而非不受控的语义或几何因素。
原文摘要 · Abstract (English)
Evaluating the performance of visual perception systems for autonomous driving is essential to ensure reliable operation across diverse environmental scenarios. Ideally, a balanced and fair analysis across different adverse conditions would require perfectly paired images of the same scene under different weather or illumination changes. This would allow evaluating the effect of photometric shifts independently of geometry and semantic changes. Unfortunately, real-world datasets rarely provide images of the same scene under different environmental conditions, because, normally, camera pose, traffic, and locations of dynamic objects (vehicles, pedestrians, etc.) vary over time, thus yielding only coarsely paired data. To address this challenge, this work introduces a data generation framework based on a high-fidelity game engine for extracting perfectly paired images. By leveraging software APIs that communicate with the GTA game engine, the framework modifies illumination and weather conditions while preserving scene geometry, camera pose, and the identity and placement of dynamic objects. For each sampled location, it procedurally instantiates dynamic entities and renders pixel-aligned images under diverse adverse conditions. The benefit of the proposed generation framework in driving scenarios is demonstrated through a systematic analysis of semantic segmentation models, whose output degradation can be attributed more directly to photometric shifts rather than to uncontrolled semantic or geometric factors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。