通过可学习校准提升扩散模型推理速度，大幅减少误差积累。
LearniBridge: Learnable Calibration of Feature Caching for Diffusion Models Acceleration
- 设计轻量级LoRA更新机制，跨时间步动态校准缓存特征。
- 在FLUX、HunyuanVideo等模型上实现最高5.87倍加速。
- 仅需3-5个样本训练，适合快速部署的生成应用。
扩散变换器（DiTs）在图像与视频生成中取得显著进展，但推理成本过高。特征缓存通过复用中间表示加速推理，现有方法依赖历史特征，导致高加速比下误差累积严重。本文研究特征校准的本质，发现最优校准更新具有跨提示共享的低秩子空间特性。基于此，提出LearniBridge：一种可学习的特征缓存校准机制，通过轻量级LoRA更新连接多个时间步。该机制仅需3-5个训练样本即可有效校准。在图像与视频生成任务上的实验表明，LearniBridge在FLUX、HunyuanVideo和WAN2.1上分别实现最高5.87×、5.75×和4.10×加速。在WAN2.1上，以4.10×加速率相较此前最先进方法提升VBench评分1.28%。代码已开源。
原文摘要 · Abstract (English)
Diffusion Transformers (DiTs) have driven substantial progress in image and video generation but suffer from prohibitive computational costs. Feature caching accelerates inference by reusing intermediate representations. Existing methods rely on historical features for implementation simplicity, yet suffer from severe error accumulation at high acceleration ratios. To address this limitation, we investigate the nature of the requisite feature correction. We demonstrate that the optimal calibration update is characterized by a shared low-rank subspace across diverse prompts. Guided by this structural insight, we propose LearniBridge, a learnable calibration mechanism for feature caching that bridges multiple timesteps through lightweight LoRA updates. This mechanism enables effective calibration requiring only 3-5 training samples. Extensive experiments on image and video generation show that LearniBridge achieves up to $5.87\times$, $5.75\times$, and $4.10\times$ acceleration on FLUX, HunyuanVideo, and WAN2.1, respectively. On WAN2.1, it improves VBench by 1.28% over the previous SOTA at $4.10\times$ acceleration. Our code is available at https://github.com/Iiiiiiirene/LearniBridge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。