用扩散模型生成多物体一致阴影,提升图像合成真实感。
MultiShadow: Multi-Object Shadow Generation for Image Compositing via Diffusion Model
- 结合图像与文本路径,通过交叉注意力融合多物体阴影位置信息。
- 在DESOBAv2基础上构建多物体合成数据集,实现更优的阴影一致性。
- 适用于需要多对象合成的视觉设计、影视特效等场景。
真实感阴影生成对实现无缝图像合成至关重要,但现有方法多聚焦单物体插入,难以推广至多物体合成场景。实际应用中,现代合成流程常需同时插入多个前景物体,要求阴影在几何、附着关系和位置上保持一致。本文解决这一未被充分研究的多物体阴影生成问题,旨在为多个插入物体生成物理上合理的阴影。方法利用预训练文本到图像扩散模型的多模态能力:图像路径注入密集多尺度特征以提供细粒度空间引导,文本路径将每个物体的阴影边界框编码为学习的位置标记,并通过交叉注意力融合;注意力对齐损失进一步将标记锚定至对应阴影区域。为支持该任务,我们在DESOBAv2数据集基础上,构建包含多个插入物体的合成场景,并自动生成结合物体类别与阴影位置信息的提示词。实验表明,该方法在单物体与多物体阴影生成设置下均达到当前最优性能。
原文摘要 · Abstract (English)
Realistic shadow generation is crucial for achieving seamless image compositing, yet existing methods primarily focus on single-object insertion and often fail to generalize when multiple foreground objects are composited into a background scene. In practice, however, modern compositing pipelines and real-world applications often insert multiple objects simultaneously, necessitating shadows that are jointly consistent in terms of geometry, attachment, and location. In this paper, we address the under-explored problem of multi-object shadow generation, aiming to synthesize physically plausible shadows for multiple inserted objects. Our approach exploits the multimodal capabilities of a pre-trained text-to-image diffusion model. An image pathway injects dense, multi-scale features to provide fine-grained spatial guidance, while a text-based pathway encodes per-object shadow bounding boxes as learned positional tokens and fuses them via cross-attention. An attention-alignment loss further grounds these tokens to their corresponding shadow regions. To support this task, we augment the DESOBAv2 dataset by constructing composite scenes with multiple inserted objects and automatically derive prompts combining object category and shadow positioning information. Experimental results demonstrate that our method achieves state-of-the-art performance in both single and multi-object shadow generation settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。