通过检测预测分歧，动态调整信任度，实现更安全的扩散模型加速。
Disagree to Accelerate: Closing the Loop on Diffusion Feature Forecasts
- 用预测结果之间的分歧判断可靠性，实时调节信任程度。
- 在相同评估次数下，多个模型生成质量显著优于现有方法。
- 无需训练，适配各类预测器，尤其能挽救低质量预测的性能。
训练无关的特征预测可跳过去噪步骤以加速扩散采样。现有工作主要聚焦于设计更强的预测器，但预测误差在不同步骤间波动剧烈，而开环缓存对每个跳过的步骤都完全信任预测结果。这种固定信任机制在高加速时会失效。关键问题不仅是如何预测得更好，更是何时以及多大程度上信任预测。本文发现，可靠性可从缓存自身观察到：当特征轨迹平滑时，两个预测一致；当预测困难时，二者分歧明显。这种分歧是低成本的运行时信号，无需额外去噪器计算。基于此，提出RACER——一种无需训练的闭环控制器，具有两种响应策略：持续将不确定预测向最近计算的特征收缩；在最危险步骤中刷新特征，并通过跳过后续预定步骤来补偿新增评估成本。推导了收缩过程的确定性误差边界，并实证验证其有效性与紧致性。在相同去噪器评估次数下，RACER在SD3.5-Large、FLUX.1-dev、Wan2.1-14B和HunyuanVideo上均超越最强的开环基线，在DrawBench、VBench和COCO上表现优异。在SD3.5上进一步证明，同等质量下采样更快。RACER可泛化至不同预测设计，例如能恢复泰勒基线丢失的大量质量。结果表明，可靠加速不仅依赖预测精度，更取决于使用方式。代码已开源。
原文摘要 · Abstract (English)
Training-free feature forecasting accelerates diffusion sampling by predicting features at skipped denoising steps. Recent work has mainly focused on designing stronger forecasters. Yet forecast error varies sharply across steps, and open-loop caches trust the forecast in full at every skipped step. This fixed trust is what breaks as acceleration turns aggressive. The missing question is not only how to forecast better, but when and how much to trust a forecast. We show that reliability can be observed from the cache itself. Two forecasts agree where the feature trajectory is smooth, and they diverge where prediction turns hard. Their disagreement is a cheap runtime signal, and it costs no extra denoiser evaluation. Based on this signal, we introduce RACER, a training-free closed-loop controller with two responses. It continuously shrinks uncertain forecasts toward the last computed feature. At the riskiest steps, RACER refreshes the feature and repays the added evaluation by skipping a later scheduled one. We derive a deterministic error bound for the shrinkage and empirically evaluate its validity and tightness across acceleration regimes. At the same number of denoiser evaluations, RACER improves the strongest open-loop baseline across SD3.5-Large, FLUX.1-dev, Wan2.1-14B, and HunyuanVideo on DrawBench, VBench, and COCO. On SD3.5, we further show that RACER samples faster at equal quality. RACER generalizes across forecasting designs as well. For example, it recovers much of the quality lost on a Taylor base. These results show that reliable diffusion acceleration also depends on how forecasts are used. Code is available at https://github.com/LiZaiyuan0619/RACER
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。