arXiv:2605.16949cs.CV2026-05被引 1

提出结构对齐方法sREPA,提升扩散Transformer的生成速度与质量。

Beyond Point-Wise Matching: Structural Representation Alignment for Accelerating Diffusion Transformers

论文配图:Beyond Point-Wise Matching: Structural Representation Alignment for Accelerating Diffusion Transformers
图 1 · 摘自论文原文
  • 用结构约束替代点对点匹配,显式建模特征图的空间关系
  • 相比现有方法,收敛更快更稳定,生成样本质量更高
  • 适合追求高效高质图像生成的开发者与研究者

近期扩散Transformer(DiTs)的研究表明,将噪声隐状态与预训练语义特征对齐(如REPA方法)可显著加速训练并提升生成保真度。后续分析(如iREPA)指出,这些优势主要源于从预训练视觉表征中迁移空间结构。然而,现有对齐方法多采用点对点匹配目标或依赖隐式架构调整,未能显式建模视觉基础模型中固有的空间关系几何结构。我们认为,这种逐元素的监督不足以捕捉视觉表征丰富的空间拓扑,有效的生成对齐应作为显式结构约束来建模。为此,我们提出sREPA——一种结构化表征对齐框架,通过强制特征图的相对几何关系一致,而非仅匹配单个特征点。该方法促使模型内化预训练特征中的整体空间布局与结构相关性,相比当前最优对齐策略,实现更快更稳定的收敛及更高的样本质量。代码与模型将公开发布。

原文摘要 · Abstract (English)

Recent advances in Diffusion Transformers (DiTs) demonstrate that aligning noisy latent states with well-trained semantic features-as pioneered by Representation Alignment (REPA)-can substantially accelerate training and improve generation fidelity. Subsequent analysis(e.g., iREPA) suggests that these gains arise primarily from transferring spatial structure contained in pre-trained vision representations. However, mostly existing alignment methods employ point-wise matching objectives or rely on implicit architectural tweaks, which fail to explicitly model the spatial relational geometry inherent in vision foundation models. We argue that such element-wise supervision is insufficient to capture the rich spatial topology of visual representations, and that effective alignment for generation should instead be formulated as an explicit structural constraint. To this end, we propose sREPA, a structural REPresentation Alignment framework to enforce consistency in the relational geometry of feature maps, rather than merely matching individual feature points. By encouraging the model to internalize holistic spatial layouts and structural correlations from pre-trained features, sREPA achieves faster and more stable convergence, along with improved sample quality, compared to state-of-the-art alignment strategies. Our code and models will be released.

扩散模型结构对齐图像生成Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。