arXiv:2509.16447cs.LG2025-09被引 3

发现扩散模型生成新组合图像的关键是局部依赖机制。

Local Mechanisms of Compositional Generalization in Conditional Diffusion

  • 提出局部条件得分理论,解释模型如何组合新条件。
  • 实验证明成功泛化的模型具备局部得分结构,失败的则没有。
  • 可应用于图像生成、大模型设计,适合研究生成机制者。

条件扩散模型具备组合泛化能力,即能生成训练中未见的条件组合样本,但其内在机制仍不明确。我们以可控的CLEVR场景研究长度泛化——生成训练时未出现的物体数量。实验发现部分模型可实现长度泛化,部分则不能,表明模型并非总能学习组合结构。我们进一步探讨局部性作为结构性机制的作用。先前工作在无条件扩散模型中提出得分局部性促进创造力,但未涉及灵活条件或组合泛化。本文证明特定组合结构(条件投影组合)与像素和条件器间稀疏依赖的局部条件得分之间存在精确等价关系,该理论亦可扩展至特征空间组合性。实证验证:在CLEVR上成功实现长度泛化的模型表现出局部条件得分,失败模型则不具备;通过因果干预显式强制局部条件得分,可使原失败模型恢复长度泛化能力。最后考察SDXL,在像素空间中存在空间局部性但缺乏条件局部性;但在网络学到的特征空间中,存在局部条件得分的定量证据。

原文摘要 · Abstract (English)

Conditional diffusion models appear capable of compositional generalization, i.e., generating convincing samples for out-of-distribution combinations of conditioners, but the mechanisms underlying this ability remain unclear. To make this concrete, we study length generalization, the ability to generate images with more objects than seen during training. In a controlled CLEVR setting (Johnson et al., 2017), we find that length generalization is achievable in some cases but not others, suggesting that models only sometimes learn the underlying compositional structure. We then investigate locality as a structural mechanism for compositional generalization. Prior works proposed score locality as a mechanism for creativity in unconditional diffusion models (Kamb & Ganguli, 2024; Niedoba et al., 2024), but did not address flexible conditioning or compositional generalization. In this paper, we prove an exact equivalence between a specific compositional structure ("conditional projective composition") (Bradley et al., 2025) and scores with sparse dependencies on both pixels and conditioners ("local conditional scores"). This theory also extends to feature-space compositionality. We validate our theory empirically: CLEVR models that succeed at length generalization exhibit local conditional scores, while those that fail do not. Furthermore, we show that a causal intervention explicitly enforcing local conditional scores restores length generalization in a previously failing model. Finally, we investigate SDXL and find that in pixel-space, spatial locality is present but conditional-locality is mostly absent; however, we find quantitative evidence of local conditional scores in the network's learned feature-space.

扩散模型组合泛化局部性生成机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。