无需额外模型,通过图像内与跨图像的特征关系匹配,加速扩散模型训练并提升生成质量。
SPARE: Structural Parameter-Free Affinity Regularization for Flow Matching

- 利用数据隐空间中特征对之间的相似性关系作为目标,无需外部编码器或投影头。
- 在相同训练预算下,比现有无参正则化方法更优,FID降低37%至54%。
- 适合追求高效训练且不想增加参数或计算开销的研究者和工程师。
去噪扩散变换器虽生成质量高,但训练收敛慢。现有正则化方法分为两类:有目标方法依赖外部特征和投影头,成本高;无目标方法仅排斥自身特征,忽略数据结构。研究表明,空间结构比全局语义更能带来性能提升。本文提出结构无参亲和力正则化(SPARE),直接以清洁数据隐空间中特征对间的相似性关系为目标,该关系是跨特征空间可比的标量,无需投影。SPARE将中间特征对的关系匹配到清洁隐空间,并扩展至跨图像的特征对匹配,同时用统一目标校准两种关系。在使用SiT骨干网络、256×256分辨率的ImageNet上,40万次迭代预算下,SPARE不引入额外编码器、头或参数,仅增加0.08 GB训练内存,却在所有测试设置中达到最低的FID,恢复了REPA 37%至54%的FID下降效果,并与REPA结合后在100万次迭代下实现1.90的FID(无分类器引导)。
原文摘要 · Abstract (English)
Denoising diffusion transformers achieve strong generation quality but converge slowly during training. Regularizing their internal representations has emerged as an effective accelerator, yet existing methods split into two families with complementary costs. Target-based methods strengthen representations by aligning them to external features, which requires an external encoder and a learnable projection head to bridge feature spaces. Target-free methods hold no reference at all, and can only repel the model's own features across samples or layers, discarding whatever structure the data contains. Prior work suggests that spatial structure, rather than global semantics, drives the gains of alignment. We therefore ask whether such structure can serve as a target directly, and whether it exists not only within an image but across images. Our key insight is that the clean data latent already carries this structure in the relations among its tokens, where a relation is the similarity between two tokens, a single scalar comparable across feature spaces without a projection head. We propose Structural Parameter-free Affinity Regularization (SPARE), a regularizer that matches the pairwise affinities of intermediate tokens to those of the clean latents. To exploit this structure fully, SPARE extends the matching to token pairs across images, precisely the pairs that prior target-free methods repel by default, and calibrates both relation types with a single learning objective. On ImageNet $256 \times 256$ with SiT backbones under matched 400K-iteration budgets, SPARE adds no encoder, head, or parameters and only 0.08 GB of training memory, yet attains the lowest FID among parameter-free regularizers in every tested setting, recovers 37 to 54\% of REPA's FID reduction, and improves over REPA when combined with it, reaching FID 1.90 under classifier-free guidance at 1M iterations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。