arXiv:2609.05981cs.CVcs.LG2026-09

用高斯过程修正扩散模型缓存误差,提速同时提升生成质量。

Accelerating Diffusion Transformers with Gaussian Process Rectified Feature Cache

论文配图:Accelerating Diffusion Transformers with Gaussian Process Rectified Feature Cache
图 1 · 摘自论文原文
  • 通过高斯过程建模缓存特征偏差,实现动态修正。
  • 结合不确定性自适应策略,仅在必要时触发全量计算,降低19.3%负载。
  • 可插拔集成现有加速方法,适合追求高效高质生成的开发者。

扩散变压器已成为生成式AI的主流范式,但其高昂的计算成本严重制约实时应用。基于预测的特征缓存广泛用于加速扩散变压器,但随着推理步数增加,预测值与完整计算轨迹之间的偏差逐渐增大。直观思路是使用在线回归模型动态校正偏差,但加速过程中缺乏标签数据。本文发现:使用缓存方法的完整计算步特征与参考全量计算轨迹之间的残差在局部呈现零均值高斯分布。将完整计算步特征视为参考特征的噪声观测,解决了数据获取难题。基于此,提出即插即用的GP-Refiner校正框架,利用高斯过程回归进行校正,并借助高斯过程回归特性,引入不确定性自适应计算策略,通过实时监控后验方差判断是否需要触发全量计算校准。实验表明,该框架与多种先进方法结合后均有显著提升。与TaylorSeer集成后,计算负载降低19.3%,PSNR提升0.9 dB,LPIPS从0.46降至0.29。代码已开源:https://github.com/Aredstone/GP-Refiner。

原文摘要 · Abstract (English)

Diffusion Transformers have become the dominant paradigm in generative AI, but their high computational costs severely hinder real-time applications. Prediction-based feature caching is widely used to accelerate diffusion transformers; however, as the number of steps increases, the deviation between its predictions and the reference full-compute trajectory gradually grows. An intuitive idea is to use an online regression model to dynamically correct this deviation, but it faces the issue of label data being unavailable during the acceleration process. This paper presents a statistical observation that the residuals between the features of full computation steps using caching methods and reference full-compute trajectory locally exhibit a zero-mean Gaussian distribution. By treating the features of full computation steps as noisy observations of reference features, the data acquisition problem is resolved. Based on this observation, a plug-and-play GP-Refiner correction framework is proposed. This method utilizes Gaussian Process Regression for correction and, leveraging the properties of GPR, introduces an uncertainty-adaptive computation strategy that triggers necessary full-computation calibration by monitoring the posterior variance in real time. Experiments demonstrate significant improvements across different models when combined with various state-of-the-art methods. Integrating the proposed framework with TaylorSeer reduces the computational load by 19.3% while improving PSNR by 0.9 dB and reducing LPIPS from 0.46 to 0.29. Code is available in https://github.com/Aredstone/GP-Refiner.

扩散模型加速推理高斯过程生成质量

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。