用数学工具让扩散模型一步生成,速度更快还保真。
One-Step Offline Distillation of Diffusion-based Models via Koopman Modeling
- 基于柯普曼理论构建线性动态模型,将复杂采样压缩为一步。
- 在多个基准上表现接近原模型,生成质量高且语义一致。
- 适合需要快速生成、追求效率的图像生成场景。
基于扩散的生成模型性能卓越,但其迭代采样过程计算开销大。离线蒸馏是降低代价的有效策略,具有高效、模块化和灵活的优势。本文提出两个关键观察:(1) 扩散模型可视为动力系统,而柯普曼理论等经典工具尚未被充分挖掘;(2) 扩散模型在隐空间中自然产生结构化、语义连贯的轨迹。基于此,我们提出柯普曼蒸馏模型(KDM),一种基于柯普曼理论的新型离线蒸馏方法——该理论可在变换空间中将非线性动态线性化。KDM 将噪声输入编码至嵌入空间,通过学习的线性算子推进状态演化,再经解码器重建干净样本,实现单步生成并保持语义一致性。理论证明:(1) 在温和假设下,学习到的扩散动态具有有限维柯普曼表示;(2) 柯普曼隐空间中的相近点对应生成输出的语义相似性,支持有效轨迹对齐。KDM 在标准离线蒸馏基准上达到高度竞争力的表现。
原文摘要 · Abstract (English)
Diffusion-based generative models have demonstrated exceptional performance, yet their iterative sampling procedures remain computationally expensive. A prominent strategy to mitigate this cost is distillation, with offline distillation offering particular advantages in terms of efficiency, modularity, and flexibility. In this work, we identify two key observations that motivate a principled distillation framework: (1) while diffusion models have been viewed through the lens of dynamical systems theory, powerful and underexplored tools can be further leveraged; and (2) diffusion models inherently impose structured, semantically coherent trajectories in latent space. Building on these observations, we introduce the Koopman Distillation Model (KDM), a novel offline distillation approach grounded in Koopman theory - a classical framework for representing nonlinear dynamics linearly in a transformed space. KDM encodes noisy inputs into an embedded space where a learned linear operator propagates them forward, followed by a decoder that reconstructs clean samples. This enables single-step generation while preserving semantic fidelity. We provide theoretical justification for our approach: (1) under mild assumptions, the learned diffusion dynamics admit a finite-dimensional Koopman representation; and (2) proximity in the Koopman latent space correlates with semantic similarity in the generated outputs, allowing for effective trajectory alignment. KDM achieves highly competitive performance across standard offline distillation benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。