用赫尔米特多项式提升扩散模型缓存预测精度,实现5.55倍加速
HiCache: A Plug-in Scaled-Hermite Upgrade for Taylor-Style Cache-then-Forecast Diffusion Acceleration
- 基于赫尔米特多项式构建特征缓存,数学匹配特征演化规律
- 在FLUX.1-dev上达5.55倍加速,图像质量不降反升
- 可插件式集成到现有缓存方法,适合生成类任务优化
扩散模型在内容生成中表现卓越,但因迭代采样带来高昂计算成本。近期特征缓存方法通过时间外推加速推理,却常因对特征演化复杂动态建模不准导致质量下降。本文提出HiCache(基于赫尔米特多项式的特征缓存),一种无需训练的加速框架,通过将数学工具与经验特性对齐提升特征预测。核心洞察是扩散Transformer中的特征导数近似呈现多变量高斯特性,因此采用赫尔米特多项式作为高斯相关过程的最优基。进一步引入双尺度机制,在保持数值稳定的同时保障预测精度,且可独立使用或与TaylorSeer结合。大量实验表明,HiCache在FLUX.1-dev上实现5.55倍加速,性能匹配或超越基线,且在文本到图像、视频生成、超分辨率任务中均表现强劲。此外,可自然嵌入已有缓存方法,如将ClusCa的图像奖励从0.9480提升至0.9840。代码已开源。
原文摘要 · Abstract (English)
Diffusion models have achieved remarkable success in content generation but often incur prohibitive computational costs due to iterative sampling. Recent feature caching methods accelerate inference via temporal extrapolation, yet can suffer quality degradation from inaccurate modeling of the complex dynamics of feature evolution. We propose HiCache (Hermite Polynomial-based Feature Cache), a training-free acceleration framework that improves feature prediction by aligning mathematical tools with empirical properties. Our key insight is that feature-derivative approximations in diffusion Transformers exhibit multivariate Gaussian characteristics, motivating the use of Hermite polynomials as a potentially optimal basis for Gaussian-correlated processes. We further introduce a dual-scaling mechanism that ensures numerical stability while preserving predictive accuracy, and is also effective when applied standalone or integrated with TaylorSeer. Extensive experiments demonstrate HiCache's superiority, achieving 5.55x speedup on FLUX.1-dev while matching or exceeding baseline quality, and maintaining strong performance across text-to-image, video generation, and super-resolution tasks. Moreover, HiCache can be naturally added to previous caching methods to enhance their performance, e.g., improving ClusCa from 0.9480 to 0.9840 in terms of image rewards. Code: https://github.com/fenglang918/HiCache
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。