arXiv:2505.20471cs.CVcs.AI2025-05AAAI被引 13

可控天气编辑技术,让3D场景中的雨雪雾效果真实且可调。

WeatherEdit: Controllable Weather Editing with 4D Gaussian Field

  • 用统一适配器整合多种天气风格,生成2D背景的多样天气。
  • 通过时序-视角注意力机制实现多帧多视角图像的一致性编辑。
  • 基于物理模拟的4D高斯场生成粒子,支持精确控制天气强度。

本文提出WeatherEdit,一种用于生成可控类型与强度的3D场景真实天气效果的新方法。该方法包含两个核心组件:天气背景编辑与天气粒子构建。在背景编辑中,引入一体化适配器,将多种天气风格集成至单一预训练扩散模型,实现2D图像背景中多样化天气生成;推理阶段设计了时序-视角(TV-)注意力机制,按特定顺序聚合时空信息,确保多帧多视角图像间的一致性。在粒子构建阶段,首先利用编辑后的图像重建3D场景,再引入动态4D高斯场生成雪花、雨滴和雾,其属性与运动通过物理驱动建模与仿真精确控制,实现逼真表现与灵活强度调节。最终将4D高斯场与3D场景融合,渲染出一致且高度真实的天气效果。在多个自动驾驶数据集上的实验表明,WeatherEdit能生成多样且可控严重程度的天气效果,展现出在恶劣天气自动驾驶模拟中的潜力。

原文摘要 · Abstract (English)

In this work, we present WeatherEdit, a novel weather editing pipeline for generating realistic weather effects with controllable types and severity in 3D scenes. Our approach is structured into two key components: weather background editing and weather particle construction. For weather background editing, we introduce an all-in-one adapter that integrates multiple weather styles into a single pretrained diffusion model, enabling the generation of diverse weather effects in 2D image backgrounds. During inference, we design a Temporal-View (TV-) attention mechanism that follows a specific order to aggregate temporal and spatial information, ensuring consistent editing across multi-frame and multi-view images. To construct the weather particles, we first reconstruct a 3D scene using the edited images and then introduce a dynamic 4D Gaussian field to generate snowflakes, raindrops and fog in the scene. The attributes and dynamics of these particles are precisely controlled through physical-based modelling and simulation, ensuring realistic weather representation and flexible severity adjustments. Finally, we integrate the 4D Gaussian field with the 3D scene to render consistent and highly realistic weather effects. Experiments on multiple driving datasets demonstrate that WeatherEdit can generate diverse weather effects with controllable condition severity, highlighting its potential for autonomous driving simulation in adverse weather. See project page: https://jumponthemoon.github.io/w-edit

3D生成天气编辑4D高斯

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。