arXiv:2602.19506cs.CVcs.LG2026-02中稿 · ICLR被引 3

通过输入输出关系提升扩散模型特征缓存精度,加速推理。

Relational Feature Caching for Accelerating Diffusion Transformers

  • 利用输入特征预测输出变化量,改进缓存预测
  • 在10个DiT模型上提速20%-45%,误差降低30%以上
  • 适合追求推理速度的生成模型部署者

特征缓存方法通过存储扩散变换器(DiTs)中计算成本高的模块在特定时间步的输出特征,用于后续步骤以减少重复计算。近期基于预测的缓存方法采用时间外推技术近似输出特征。尽管有效,但完全依赖时间外推仍存在显著预测误差,导致性能下降。通过详细分析发现:1)误差源于输出特征变化幅度的不规则性;2)模块的输入特征与对应输出高度相关。基于此,我们提出关系特征缓存(RFC),一种新框架,利用输入-输出关系提升特征预测准确性。具体地,引入关系特征估计(RFE),从输入特征估算输出特征的变化量,实现更精确预测;同时提出关系缓存调度(RCS),利用输入特征估计预测误差,在误差较大时执行完整计算。在多个DiT模型上的实验表明,RFC显著优于现有方法。

原文摘要 · Abstract (English)

Feature caching approaches accelerate diffusion transformers (DiTs) by storing the output features of computationally expensive modules at certain timesteps, and exploiting them for subsequent steps to reduce redundant computations. Recent forecasting-based caching approaches employ temporal extrapolation techniques to approximate the output features with cached ones. Although effective, relying exclusively on temporal extrapolation still suffers from significant prediction errors, leading to performance degradation. Through a detailed analysis, we find that 1) these errors stem from the irregular magnitude of changes in the output features, and 2) an input feature of a module is strongly correlated with the corresponding output. Based on this, we propose relational feature caching (RFC), a novel framework that leverages the input-output relationship to enhance the accuracy of the feature prediction. Specifically, we introduce relational feature estimation (RFE) to estimate the magnitude of changes in the output features from the inputs, enabling more accurate feature predictions. We also present relational cache scheduling (RCS), which estimates the prediction errors using the input features and performs full computations only when the errors are expected to be substantial. Extensive experiments across various DiT models demonstrate that RFC consistently outperforms prior approaches significantly. Project page is available at https://cvlab.yonsei.ac.kr/projects/RFC

扩散模型特征缓存推理加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。