用单步扩散模型生成可控阴影，无需3D几何信息
Controllable Shadow Generation with Single-Step Diffusion Models from Synthetic Data
- 基于合成数据训练单步扩散模型，直接生成可调阴影
- 仅需一步采样即可实现高质量阴影，支持实时应用
- 模型能泛化到真实图像，适合影视特效与图像合成
真实感阴影生成是高质量图像合成与视觉特效的关键，但现有方法存在局限：基于物理的方法需要3D场景几何，常不可得；学习类方法则控制性差且易产生视觉伪影。本文提出一种快速、可控、背景无关的2D物体阴影生成新方法。通过3D渲染引擎构建大规模合成数据集，训练扩散模型以生成不同光照参数下的阴影图。大量消融实验表明，使用修正流目标函数可在仅一步采样下实现高质量结果，支持实时应用。此外，实验验证了模型对真实图像的良好泛化能力。为促进阴影生成的质量与可控性评估，我们发布了包含多样物体与多种光照设置的公开基准数据集。项目页面见 https://gojasper.github.io/controllable-shadow-generation-project/
原文摘要 · Abstract (English)
Realistic shadow generation is a critical component for high-quality image compositing and visual effects, yet existing methods suffer from certain limitations: Physics-based approaches require a 3D scene geometry, which is often unavailable, while learning-based techniques struggle with control and visual artifacts. We introduce a novel method for fast, controllable, and background-free shadow generation for 2D object images. We create a large synthetic dataset using a 3D rendering engine to train a diffusion model for controllable shadow generation, generating shadow maps for diverse light source parameters. Through extensive ablation studies, we find that rectified flow objective achieves high-quality results with just a single sampling step enabling real-time applications. Furthermore, our experiments demonstrate that the model generalizes well to real-world images. To facilitate further research in evaluating quality and controllability in shadow generation, we release a new public benchmark containing a diverse set of object images and shadow maps in various settings. The project page is available at https://gojasper.github.io/controllable-shadow-generation-project/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。