让罕见概念生成更精准，通过注意力与正交补提升文本到图像质量
ADAPT: Attention Driven Adaptive Prompt Scheduling and InTerpolating Orthogonal Complements for Rare Concepts Generation
- 基于注意力分数和正交补，确定性规划提示调度
- 在RareBench上显著提升罕见概念组合生成效果
- 无需训练即可实现稳定可控的罕见属性生成
文本到图像生成中，罕见组合概念的生成仍是扩散模型的挑战，尤其当属性在训练数据中不常见时。现有方法如R2F虽利用大语言模型进行提示调度，但受语言模型随机性影响，且迭代文本嵌入切换带来次优引导。为此，我们提出ADAPT框架——一种无需训练的确定性方法，通过注意力得分与正交成分，对提示调度进行语义对齐与规划，增强罕见概念的组合生成。在RareBench基准上，ADAPT在不增加额外训练或微调的前提下，显著提升了罕见概念生成性能，准确反映罕见属性语义信息，实现确定性、高精度控制，同时保持视觉完整性。
原文摘要 · Abstract (English)
Generating rare compositional concepts in text-to-image synthesis remains a challenge for diffusion models, particularly for attributes that are uncommon in the training data. While recent approaches, such as R2F, address this challenge by utilizing LLM for prompt scheduling, they suffer from inherent variance due to the randomness of language models and suboptimal guidance from iterative text embedding switching. To address these problems, we propose the ADAPT framework, a training-free framework that deterministically plans and semantically aligns prompt schedules, providing consistent guidance to enhance the composition of rare concepts. By leveraging attention scores and orthogonal components, ADAPT significantly enhances compositional generation of rare concepts in the RareBench benchmark without additional training or fine-tuning. Through comprehensive experiments, we demonstrate that ADAPT achieves superior performance in RareBench and accurately reflects the semantic information of rare attributes, providing deterministic and precise control over the generation of rare compositions without compromising visual integrity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。