让文生图模型可参数化控制人物阴影，无需真实光照数据
Parametric Shadow Control for Portrait Generation in Text-to-Image Diffusion Models
- 通过小网络提取扩散模型中的隐含阴影属性
- 仅需数千张合成图像和数小时训练，无需真实光场数据
- 支持阴影形状、位置、强度的直观调节，适配多种艺术风格
文生图扩散模型在生成多样化人像方面表现优异，但缺乏直观的阴影控制能力。现有编辑方法作为后处理手段，在跨风格操控上效果有限，且通常依赖昂贵的真实世界光场数据采集或高计算资源训练。为此，我们提出Shadow Director，一种从已训练扩散模型中提取并操纵隐藏阴影属性的方法。该方法仅需一个小型估计网络，基于数千张合成图像进行数小时训练，无需真实光场数据。Shadow Director可在人像生成过程中实现对阴影形状、位置和强度的参数化与直观控制，同时保持艺术风格一致性和身份特征。尽管训练仅基于真实身份的合成数据，仍能有效泛化至多种风格的人像生成，是一种更易用、资源友好的解决方案。
原文摘要 · Abstract (English)
Text-to-image diffusion models excel at generating diverse portraits, but lack intuitive shadow control. Existing editing approaches, as post-processing, struggle to offer effective manipulation across diverse styles. Additionally, these methods either rely on expensive real-world light-stage data collection or require extensive computational resources for training. To address these limitations, we introduce Shadow Director, a method that extracts and manipulates hidden shadow attributes within well-trained diffusion models. Our approach uses a small estimation network that requires only a few thousand synthetic images and hours of training-no costly real-world light-stage data needed. Shadow Director enables parametric and intuitive control over shadow shape, placement, and intensity during portrait generation while preserving artistic integrity and identity across diverse styles. Despite training only on synthetic data built on real-world identities, it generalizes effectively to generated portraits with diverse styles, making it a more accessible and resource-friendly solution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。