通过分层加速策略,让世界模型生成更快且不丢质量。
HERO: Hierarchical Extrapolation and Refresh for Efficient World Models

- 浅层用局部重采样,深层用线性外推,跳过冗余计算。
- 实测速度提升1.73倍,画质损失极小。
- 适合追求高效生成的虚拟环境开发人员。
生成式世界模型能构建沉浸式虚拟环境,但因扩散模型的迭代特性导致推理缓慢。尽管近期扩散模型效率有所提升,直接应用于世界模型会引发画质下降等问题。本文提出HERO——一种无需训练的分层加速框架。由于世界模型具有多模态特性,我们发现浅层特征随时间变化剧烈,深层特征则更稳定。基于此,HERO采用分层策略:(i) 浅层使用块级重置机制,通过局部采样与频率感知追踪,高效选择需重计算的令牌,无需额外度量计算且兼容FlashAttention;(ii) 深层采用线性外推方案,直接预测中间特征,完全跳过注意力与前馈网络的计算。实验表明,HERO在仅带来微小质量损失的前提下实现1.73倍加速,显著优于现有扩散模型加速方法。
原文摘要 · Abstract (English)
Generation-driven world models create immersive virtual environments but suffer slow inference due to the iterative nature of diffusion models. While recent advances have improved diffusion model efficiency, directly applying these techniques to world models introduces limitations such as quality degradation. In this paper, we present HERO, a training-free hierarchical acceleration framework tailored for efficient world models. Owing to the multi-modal nature of world models, we identify a feature coupling phenomenon, wherein shallow layers exhibit high temporal variability, while deeper layers yield more stable feature representations. Motivated by this, HERO adopts hierarchical strategies to accelerate inference: (i) In shallow layers, a patch-wise refresh mechanism efficiently selects tokens for recomputation. With patch-wise sampling and frequency-aware tracking, it avoids extra metric computation and remain compatible with FlashAttention. (ii) In deeper layers, a linear extrapolation scheme directly estimates intermediate features. This completely bypasses the computations in attention modules and feed-forward networks. Our experiments show that HERO achieves a 1.73$\times$ speedup with minimal quality degradation, significantly outperforming existing diffusion acceleration methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。