让扩散模型更准确地生成多概念图像，避免物体混淆或缺失。
Steer Away From Mode Collisions: Improving Composition In Diffusion Models
- 通过修正采样路径避开混合模式，提升多概念共现能力。
- 在多个提示下实现更高概念覆盖率与视觉平衡性，减少物体丢失或变形。
- 无需重训练，可直接集成到现有模型中，适合图像生成研究者使用。
我们提出改进文本到图像扩散模型在多概念提示下的表现。常见问题如“一只猫和一只狗”可能生成缺失、模糊或重叠的图像。我们假设这是由于模型在采样时进入混合模式，过度强调训练中学习强烈的单一概念。为此，我们引入一种无需重训练的校正采样策略——CO3，主动避开联合提示行为与任一单概念高度重合的区域，引导模型趋向所有概念均衡共存的“纯净”联合模式。我们进一步发现，现有组合引导方法在某些权重区间不稳定，会加剧不平衡；我们刻画了稳定区域并调整采样以维持其内。实验表明，相比标准基线与先前组合方法，CO3在多样多概念提示下显著提升概念覆盖度、平衡性与鲁棒性，更少出现概念被丢弃或扭曲的情况。结果表明，轻量级校正引导能有效缓解现代扩散系统中的脆弱语义对齐问题。代码已开源：https://github.com/debottam-dutta7/co3
原文摘要 · Abstract (English)
We propose to improve multi-concept prompt fidelity in text-to-image diffusion models. We begin with common failure cases - prompts like "a cat and a dog" that sometimes yields images where one concept is missing, faint, or colliding awkwardly with another. We hypothesize that this happens when the diffusion model drifts into mixed modes that over-emphasize a single concept it learned strongly during training. Instead of re-training, we introduce a corrective sampling strategy that steers away from regions where the joint prompt behavior overlaps too strongly with any single concept in the prompt. The goal is to steer towards "pure" joint modes where all concepts can coexist with balanced visual presence. We further show that existing multi-concept guidance schemes can operate in unstable weight regimes that amplify imbalance; we characterize favorable regions and adapt sampling to remain within them. Our approach, CO3, is plug-and-play, requires no model tuning, and complements standard classifier-free guidance. Experiments on diverse multi-concept prompts indicate improvements in concept coverage, balance and robustness, with fewer dropped or distorted concepts compared to standard baselines and prior compositional methods. Results suggest that lightweight corrective guidance can substantially mitigate brittle semantic alignment behavior in modern diffusion systems. Code is available at https://github.com/debottam-dutta7/co3
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。