解决布局引导生成中对象错位和属性混淆问题
MALeR: Improving Compositional Fidelity in Layout-Guided Generation
- 引入掩码属性感知绑定机制，精准控制对象位置与属性
- 有效防止无关对象出现，提升生成图像的分布一致性
- 适合多主体多属性复杂场景的高质量图像生成
近年来，文本到图像模型推动了创意与可控图像生成的新时代。然而，在包含多个主体和属性的组合场景中生成仍面临挑战。尽管已有布局引导方法提升用户对主体位置的控制，但在组合场景中仍存在诸多问题：非预期对象出现在布局外、生成图像分布异常且含不自然伪影，或属性在主体间泄露，导致视觉输出错误。本文提出MALeR，针对上述问题进行改进。给定文本提示与对应布局，该方法可防止主体出现在布局之外，同时保持生成结果在分布内。此外，我们设计了一种掩码式、属性感知的绑定机制，有效抑制属性泄露，实现多属性主体在复杂组合场景中的准确渲染。定性与定量评估表明，相比以往方法，MALeR在组合准确性、生成一致性及属性绑定方面均表现更优。尤其适用于生成包含多个主体及每个主体具多个属性的复杂场景。
原文摘要 · Abstract (English)
Recent advances in text-to-image models have enabled a new era of creative and controllable image generation. However, generating compositional scenes with multiple subjects and attributes remains a significant challenge. To enhance user control over subject placement, several layout-guided methods have been proposed. However, these methods face numerous challenges, particularly in compositional scenes. Unintended subjects often appear outside the layouts, generated images can be out-of-distribution and contain unnatural artifacts, or attributes bleed across subjects, leading to incorrect visual outputs. In this work, we propose MALeR, a method that addresses each of these challenges. Given a text prompt and corresponding layouts, our method prevents subjects from appearing outside the given layouts while being in-distribution. Additionally, we propose a masked, attribute-aware binding mechanism that prevents attribute leakage, enabling accurate rendering of subjects with multiple attributes, even in complex compositional scenes. Qualitative and quantitative evaluation demonstrates that our method achieves superior performance in compositional accuracy, generation consistency, and attribute binding compared to previous work. MALeR is particularly adept at generating images of scenes with multiple subjects and multiple attributes per subject.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。