用新方法提升扩散模型的物体分离与组合对齐，无需重训练
JEDI: The Force of Jensen-Shannon Divergence in Disentangling Diffusion Models
- 基于杰森-申农散度优化注意力图，减少语义混杂
- 仅需少量迭代步数即可实现显著改进，提升复杂场景对齐效果
- 不依赖外部模型，适合需要快速适配的生成任务
我们提出JEDI，一种测试时自适应方法，可在不重新训练或外部监督的情况下增强扩散模型的主体分离与组合对齐能力。JEDI通过基于杰森-申农散度的新目标函数，最小化注意力图中的语义混杂。为提升效率，采用对抗式优化，大幅减少所需更新步数。JEDI具有模型无关性，适用于Stable Diffusion 1.5和3.5等架构，在复杂场景中持续改善提示对齐与解耦效果。此外，JEDI提供一种轻量级、无需CLIP的解耦评分，基于内部注意力分布，为测试时条件下的组合对齐提供原则性评估基准。代码与结果见https://ericbill21.github.io/JEDI/
原文摘要 · Abstract (English)
We introduce JEDI, a test-time adaptation method that enhances subject separation and compositional alignment in diffusion models without requiring retraining or external supervision. JEDI operates by minimizing semantic entanglement in attention maps using a novel Jensen-Shannon divergence based objective. To improve efficiency, we leverage adversarial optimization, reducing the number of updating steps required. JEDI is model-agnostic and applicable to architectures such as Stable Diffusion 1.5 and 3.5, consistently improving prompt alignment and disentanglement in complex scenes. Additionally, JEDI provides a lightweight, CLIP-free disentanglement score derived from internal attention distributions, offering a principled benchmark for compositional alignment under test-time conditions. Code and results are available at https://ericbill21.github.io/JEDI/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。