通过块级自适应缓存,让扩散策略推理提速三倍且无精度损失。
Block-wise Adaptive Caching for Accelerating Diffusion Policy
- 按块动态缓存中间动作特征,根据相似性变化智能决定更新时机。
- 在多个机器人基准上实现最高3倍推理加速,精度零下降。
- 无需训练,可直接嵌入现有扩散策略模型,适合实时机器人控制场景。
扩散策略展现出强大的视觉运动建模能力,但其高计算开销使其难以用于实时机器人控制。尽管重复去噪步骤中存在巨大冗余,现有加速技术因架构和数据差异无法有效适配扩散策略。本文提出块级自适应缓存(BAC),通过缓存中间动作特征实现无损加速。BAC基于特征相似性在时间上非均匀分布、块间模式各异的观察,设计自适应缓存调度器,以最大化缓存与跳过特征间的全局相似性来确定最优更新时间步。然而,对每块独立应用该调度会导致误差在块间传播,尤其在前馈网络(FFN)块中显著上升。为此,我们提出气泡合并算法,通过在下游FFN前更新存在显著缓存误差的上游块来截断误差传播。作为无需训练的插件,BAC可无缝集成至基于Transformer的扩散策略及视觉-语言-动作模型。大量实验表明,BAC在多个机器人基准上实现高达3倍的推理加速。项目主页:https://block-wise-adaptive-caching.github.io。
原文摘要 · Abstract (English)
Diffusion Policy has demonstrated strong visuomotor modeling capabilities, but its high computational cost renders it impractical for real-time robotic control. Despite huge redundancy across repetitive denoising steps, existing diffusion acceleration techniques fail to generalize to Diffusion Policy due to fundamental architectural and data divergences. In this paper, we propose $\textbf{B}$lock-wise $\textbf{A}$daptive $\textbf{C}$aching ($\textbf{BAC}$), a method to accelerate Diffusion Policy by caching intermediate action features. BAC achieves lossless action generation acceleration by adaptively updating and reusing cached features at the block level, based on a key observation that feature similarities exhibit non-uniform temporal dynamics and distinct block-specific patterns. To operationalize this insight, we first design an Adaptive Caching Scheduler to identify optimal update timesteps by maximizing the global feature similarities between cached and skipped features. However, applying this scheduler for each block leads to significant error surges due to the inter-block propagation of caching errors, particularly within Feed-Forward Network (FFN) blocks. To mitigate this issue, we develop the Bubbling Union Algorithm, which truncates these errors by updating the upstream blocks with significant caching errors before downstream FFNs. As a training-free plugin, BAC is readily integrable with existing transformer-based Diffusion Policy and vision-language-action models. Extensive experiments on multiple robotic benchmarks demonstrate that BAC achieves up to 3$\times$ inference speedup for free. Project page: https://block-wise-adaptive-caching.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。