通过自修复机制实现低延迟图像生成,提升少步扩散模型效率。
SelfLift: Accelerating Few-Step Diffusion via Self-Recovering Resolution Transition
- 利用模型自身生成一致性信号,自动修复分辨率跃迁误差
- 在2步和3步扩散中分别降低41.5%和44.1%端到端延迟
- 无需超分网络或额外采样步骤,适合高效生成场景
少步扩散模型显著压缩了时间计算,使每一步的空間計算成本成為推理延遲的主要來源。進階分辨率推理通過在低分辨率下進行早期去噪,並將高分辨率計算保留至精修階段來降低此開銷。然而,現有方法通常直接提升中間潛在表示,依賴後續步驟吸收由此產生的分布不匹配。在少步情境下,有限的恢復預算導致這些誤差以可見偽影形式殘留,限制了轉換時機,從而影響效率。我們提出 SelfLift,一種自恢復式進階分辨率框架,從生成模型本身獲取轉換修復信號與軌跡對齊監督。SelfLift-zero 提出無訓練的「異常感知一致性躍遷」,利用直接潛在提升與像素-VAE重編碼之間的不一致作為局部偽影風險信號與模型內校正方向。該方法實現可靠晚躍遷,無需外部超分、額外去噪器評估或採樣週期修改。基於此穩健躍遷,SelfLift-rich 在學生訪問狀態上執行在策略自恢復,從內部自教師傳遞密集高分辨率指導,同時保持與變更的進階分辨率動態一致。在 FLUX.2-Klein 與 Z-Image-Turbo 上,SelfLift 分別將端到端延遲降低 41.5% 和 44.1%。結合時間步蒸餾,其相較於對應 50 步模型實現 29.61× 和 19.21× 的整體加速,並保持競爭性生成品質,建立少步擴散模型更優的效率-品質邊界。
原文摘要 · Abstract (English)
Few-step diffusion models substantially compress temporal computation, making the spatial cost of each model evaluation an increasingly dominant source of inference latency. Progressive-resolution inference reduces this cost by performing early denoising at low resolution and reserving high-resolution computation for refinement. However, existing methods typically lift intermediate latents directly and rely on subsequent steps to absorb the induced distribution mismatch. In the few-step regime, the limited recovery budget leaves these errors as visible artifacts, constraining how late the transition can occur and, consequently, how efficiently it can be performed. We introduce SelfLift, a self-recovering progressive-resolution framework that derives both transition-repair signals and trajectory-aligned supervision from the generative model itself. SelfLift-zero proposes a training-free Artifact-Aware Consistency Lift, using disagreement between direct latent lifting and pixel-VAE re-encoding as both a localized artifact-risk signal and a model-native correction direction. It enables reliable late transitions without external super-resolution, extra denoiser evaluations, or sampling-schedule modifications. Building on this robust transition, SelfLift-rich performs On-Policy Self Recovery on student-visited states, transferring dense high-resolution guidance from an internal self-teacher while remaining aligned with the altered progressive-resolution dynamics. Across FLUX.2-Klein and Z-Image-Turbo, SelfLift reduces end-to-end latency by 41.5% and 44.1%, respectively. Combined with timestep distillation, it delivers overall speedups of 29.61x and 19.21x over the corresponding 50-step models while preserving competitive generation quality, establishing a stronger speed-quality frontier for few-step diffusion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。