让扩散Transformer在特征流形上高效生成，突破收敛瓶颈。
Learning on the Manifold: Unlocking Standard Diffusion Transformers with Representation Encoders
- 用黎曼流形匹配替代欧式路径，沿特征流形表面生成
- 标准DiT-B模型（131M参数）收敛，FID达3.37
- 适合做高效高保真生成的工程师和研究者
利用表示编码器进行生成建模可实现高效、高保真合成。然而，标准扩散Transformer无法直接在其表示空间中收敛。尽管近期工作将其归因于容量瓶颈并提出计算成本高昂的宽度扩展方法，我们证明其根本原因是几何问题。我们识别出几何干扰（Geometric Interference）为根源：标准欧氏流形匹配迫使概率路径穿越表示编码器超球面特征空间的低密度内部，而非沿流形表面行进。为此，我们提出黎曼流形匹配结合雅可比正则化（RJF），通过约束生成过程沿流形测地线，并纠正曲率引起的误差传播，使标准扩散Transformer无需宽度扩展即可收敛。我们的方法使标准DiT-B架构（131M参数）有效收敛，取得FID 3.37，而先前方法无法收敛。代码见：https://github.com/amandpkr/RJF
原文摘要 · Abstract (English)
Leveraging representation encoders for generative modeling offers a path for efficient, high-fidelity synthesis. However, standard diffusion transformers fail to converge on these representations directly. While recent work attributes this to a capacity bottleneck proposing computationally expensive width scaling of diffusion transformers we demonstrate that the failure is fundamentally geometric. We identify Geometric Interference as the root cause: standard Euclidean flow matching forces probability paths through the low-density interior of the hyperspherical feature space of representation encoders, rather than following the manifold surface. To resolve this, we propose Riemannian Flow Matching with Jacobi Regularization (RJF). By constraining the generative process to the manifold geodesics and correcting for curvature-induced error propagation, RJF enables standard Diffusion Transformer architectures to converge without width scaling. Our method RJF enables the standard DiT-B architecture (131M parameters) to converge effectively, achieving an FID of 3.37 where prior methods fail to converge. Code: https://github.com/amandpkr/RJF
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。