SARA通过分层对齐提升扩散模型训练效率与生成质量
SARA: Structural and Adversarial Representation Alignment for Training-efficient Diffusion Models
- 分层对齐：局部、结构、全局三重约束
- FID达1.36，训练速度是REPA的两倍
- 适合追求高效高质图像生成的研究者
现代扩散模型面临训练效率与生成质量的权衡。现有表示对齐方法如REPA虽通过逐块对齐加速收敛，却难以捕捉视觉表示中的结构关系并保证预训练编码器与去噪网络间全局分布一致性。为此，我们提出SARA，一种分层对齐框架，施加多层级表示约束：(1) 逐块对齐以保留局部语义细节，(2) 自相关矩阵对齐以维持表示内部结构一致性，(3) 对抗分布对齐以缓解全局表示差异。SARA显式建模自相似性矩阵中的内部相关性及对抗对齐中的跨分布一致性，实现局部与全局尺度的全面对齐。在ImageNet-256上的实验表明，SARA在达到1.36的FID的同时，收敛速度为REPA的两倍，优于近期最先进的图像生成方法。本工作建立了一种通过分层表示对齐优化扩散训练的系统性范式。
原文摘要 · Abstract (English)
Modern diffusion models encounter a fundamental trade-off between training efficiency and generation quality. While existing representation alignment methods, such as REPA, accelerate convergence through patch-wise alignment, they often fail to capture structural relationships within visual representations and ensure global distribution consistency between pretrained encoders and denoising networks. To address these limitations, we introduce SARA, a hierarchical alignment framework that enforces multi-level representation constraints: (1) patch-wise alignment to preserve local semantic details, (2) autocorrelation matrix alignment to maintain structural consistency within representations, and (3) adversarial distribution alignment to mitigate global representation discrepancies. Unlike previous approaches, SARA explicitly models both intra-representation correlations via self-similarity matrices and inter-distribution coherence via adversarial alignment, enabling comprehensive alignment across local and global scales. Experiments on ImageNet-256 show that SARA achieves an FID of 1.36 while converging twice as fast as REPA, surpassing recent state-of-the-art image generation methods. This work establishes a systematic paradigm for optimizing diffusion training through hierarchical representation alignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。