让自动驾驶视频天气切换更真实可控,无需大量数据
AutoWeather4D: Autonomous Driving Video Weather Conversion via G-Buffer Dual-Pass Editing
- 分两步处理:先建模物体结构,再动态计算光照变化
- 生成效果媲美主流模型,且可调节光照参数
- 适合需要精准控制天气的自动驾驶测试场景
生成式视频模型在合成自动驾驶恶劣天气方面取得显著进展,但通常需依赖大规模数据来学习罕见天气场景。尽管3D感知编辑方法通过扩充现有视频缓解了数据需求,却受限于每场景高昂的优化成本,并存在几何与光照耦合问题。本文提出AutoWeather4D,一种前馈式3D感知天气编辑框架,显式解耦几何与光照。核心为G-buffer双阶段编辑机制:几何阶段利用显式结构基础实现表面锚定的物理交互;光照阶段解析光传输,将局部光源贡献累积为全局光照,实现动态3D局部调光。大量实验表明,AutoWeather4D在保真度和结构一致性上达到生成基线水平,同时支持细粒度参数化物理控制,可作为自动驾驶领域实用的数据生成引擎。
原文摘要 · Abstract (English)
Generative video models have significantly advanced the photorealistic synthesis of adverse weather for autonomous driving; however, they consistently demand massive datasets to learn rare weather scenarios. While 3D-aware editing methods alleviate these data constraints by augmenting existing video footage, they are fundamentally bottlenecked by costly per-scene optimization and suffer from inherent geometric and illumination entanglement. In this work, we introduce AutoWeather4D, a feed-forward 3D-aware weather editing framework designed to explicitly decouple geometry and illumination. At the core of our approach is a G-buffer Dual-pass Editing mechanism. The Geometry Pass leverages explicit structural foundations to enable surface-anchored physical interactions, while the Light Pass analytically resolves light transport, accumulating the contributions of local illuminants into the global illumination to enable dynamic 3D local relighting. Extensive experiments demonstrate that AutoWeather4D achieves comparable photorealism and structural consistency to generative baselines while enabling fine-grained parametric physical control, serving as a practical data engine for autonomous driving.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。