arXiv:2606.29020cs.CVcs.AI2026-06被引 1

让天气视频生成更真实可控,支持多样外观与物理级粒子动态。

Semantic-Aware, Physics-Informed, Geometry-Grounded Weather Video Synthesis

论文配图:Semantic-Aware, Physics-Informed, Geometry-Grounded Weather Video Synthesis
图 1 · 摘自论文原文
  • 分三路引导:语义定外观、物理模拟动效、几何对齐位置
  • 能生成密集雨雪等重天气,且运动符合重力风力规律
  • 适合自动驾驶数据增强,提升恶劣天气下分割鲁棒性

天气合成旨在为输入视频添加天气效果,同时保持场景身份、结构和运动的一致性。现有方法的主要局限在于天气外观缺乏多样性,且难以有效控制天气动态(如时间演化和粒子运动)。多数方法依赖文本提示,但其表达模糊,常无法生成细致的天气特征。此外,通用视频编辑器为追求清晰美观输出,常抑制强天气现象,导致密集粒子效果难以生成。为此,我们提出一种语义感知、物理驱动、几何约束的框架,引导现成视频编辑器生成多样全局外观与精细粒子动态。将合成过程分解为三个条件信号:语义决定天气外观,动态控制其随时间演变,几何确定其在场景中的分布位置。具体包括:(1) 语义感知的外观锚定,基于场景语义与用户输入建立目标外观;(2) 物理驱动的动态模拟,通过重力、风力和湍流作用下的高斯表示粒子场模拟粒子效果;(3) 几何对齐的视频合成,使模拟粒子与目标场景几何对齐并生成最终视频。实验表明,本方法可生成多样、物理与视觉均真实的天气效果。进一步验证显示,合成数据显著提升了自动驾驶语义分割在恶劣天气下的鲁棒性。

原文摘要 · Abstract (English)

Weather synthesis aims to add weather effects to input videos while preserving scene identity, structure, and motion. The key limitation of existing methods is the lack of diversity in weather appearance and effective control over weather dynamics (e.g., temporal evolution and particle motion). Most approaches rely on text prompts, which are inherently underspecified and often fail to produce detailed weather characteristics. Additionally, general-purpose video editors optimized for clean and aesthetic outputs tend to suppress heavy weather phenomena, making dense particle effects difficult to generate. To address these, we propose a Semantic-Aware, Physics-Informed, and Geometry-Grounded framework that steers an off-the-shelf video editor to synthesize diverse global appearances and detailed particle dynamics. We factorize the synthesis into three conditional signals, so that each provides a distinct and stable source of guidance: semantics specifies what the weather should look like, dynamics governs how it evolves over time, and geometry determines where it should appear in the scene. Specifically, we introduce (1) semantic-aware appearance anchoring to establish the target appearance from scene semantics and user input; (2) physics-informed dynamic simulation to generate particle effects by simulating a Gaussian-represented particle field under gravity, wind, and turbulence; and (3) geometry-grounded video synthesis to align the simulated particles with target scene geometry and synthesize the final video. Experiments demonstrate that our method produces diverse, physically and visually realistic weather effects. Furthermore, we show that our synthesized data significantly improves the robustness of autonomous driving semantic segmentation under adverse weather conditions. Project page: https://jumponthemoon.github.io/w-crafter/.

天气生成物理模拟自动驾驶视频合成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。