arXiv:2605.27075cs.CV2026-05被引 1

根据预算动态刷新缓存,让扩散模型推理更快更稳。

BeCARE: Budgeted Cache Refresh for Diffusion Transformer Acceleration

论文配图:BeCARE: Budgeted Cache Refresh for Diffusion Transformer Acceleration
图 1 · 摘自论文原文
  • 用噪声调度分析误差影响,动态评估缓存风险
  • 在3.3倍加速下比最强基线提升2.7 dB图像质量
  • 无需调参即可跨不同计算预算复用,适合部署优化

训练无关的特征缓存通过重用或预测中间特征加速扩散变换器(DiT)推理。但固定刷新周期使计算可预测却忽略提示和时间步相关的误差风险;手动设定阈值虽能局部适应,却难以控制实际计算量。为此,我们提出预算式缓存刷新(BeCARE),在用户指定全计算上限的前提下,自适应决定加速推理中何时刷新缓存。首先,从采样器的噪声调度推导误差放大曲线,刻画去噪各阶段近似误差的影响差异;再结合提示特异性外推残差与缓存年龄,构建累积风险评分,当继续缓存将造成损害时触发刷新。同时,利用同一曲线建立解析支出参考,并通过反馈调整刷新阈值,实现有限刷新预算在采样轨迹上的合理分配。此外,结构化防护机制(固定预热、预算衍生的后期预留、最大缓存复用长度)防止不可靠外推和预算过早耗尽。这些设计支持提示自适应刷新位置,且单次校准即可跨预算层级迁移,无需每层单独调优。在FLUX.1-dev上,针对200个提示并测量FLOPs,本方法在3~6倍加速范围内持续优于代表性训练无关缓存基线。主运行点(3.3倍加速)下,图像质量(配对PSNR)相比最强基线提升2.7 dB,且不增加计算量。在SD3.5 Large上,亦显著优于固定间隔调度的同款泰勒缓存引擎,在匹配计算条件下表现更佳。

原文摘要 · Abstract (English)

Training-free feature caching accelerates diffusion transformer (DiT) inference by reusing or forecasting intermediate features. However, fixed schedules make compute predictable but ignore prompt- and timestep-dependent risk, whereas hand-tuned error thresholds adapt locally but leave realized compute difficult to control. To address these limitations, we present Budgeted Cache Refresh (BeCARE), a training-free framework that, given a user-specified cap on full computations, adaptively determines when to refresh the cache during accelerated inference. Specifically, we first derive an error-amplification profile from the sampler's noise schedule to characterize the varying impact of approximation errors across denoising timesteps. We then combine this profile with prompt-specific extrapolation residuals and cache age to form an accumulated risk score, triggering a refresh when continued caching becomes harmful. Meanwhile, we use the same profile to construct an analytic spending reference and adjust the refresh threshold through feedback, thereby distributing the limited refresh budget over the sampling trajectory. Finally, structural safeguards (a fixed warmup, a budget-derived late-stage reserve, and a maximum cache-reuse length) prevent unreliable extrapolation and premature budget exhaustion. These designs enable prompt-adaptive refresh placement and allow a single calibration to transfer across budget tiers without per-tier tuning. Experiments on FLUX.1-dev with 200 prompts and measured FLOPs show that our method consistently outperforms representative training-free caching baselines across speedups from 3x to 6x. At the main operating point (3.3x acceleration), it improves paired PSNR by 2.7 dB over the strongest baseline using no more compute. On SD3.5 Large, it also substantially improves the same Taylor cache engine over fixed-interval scheduling at matched compute.

扩散模型缓存优化推理加速预算控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。