通过进化搜索优化扩散模型缓存策略,显著提速且几乎不损失性能。
Evolving Cache Schedules for Fast Diffusion Policy Inference

- 用进化算法全局调度缓存刷新时机,智能跳过冗余计算。
- 在保持接近全性能前提下,最高提速8.05倍,浮点运算量降至1.96G。
- 无需重训练,可直接部署于预训练扩散策略,适合实时机器人控制。
扩散策略通过迭代去噪动作块实现强大的视觉-运动控制,但重复去噪导致实时部署计算开销大。基于缓存的方法通过复用中间激活降低推理成本,但现有免训练调度通常均匀分配计算资源,忽略各模块间冗余差异,导致性能-效率权衡不佳。为此,我们提出免训练加速框架EVO,通过进化搜索全局调度缓存刷新。EVO将每个候选方案表示为块-时间步网格上的完整调度,使迭代去噪中冗余的Transformer计算可通过缓存重用跳过,同时保持闭环推演性能。为提升搜索效率,EVO引入冗余感知初始化,以高潜力调度种子启动种群,并采用目标条件早停机制,在达到预期性能目标后终止验证。离线优化的调度可直接嵌入预训练扩散策略,无需重训练。大量操控基准测试表明,EVO在几乎保留全部性能的同时大幅减少计算,最高实现8.05倍动作生成速度提升,浮点运算量从15.77G降至最低1.96G。源码见https://github.com/pillom/EVO。
原文摘要 · Abstract (English)
Diffusion policies achieve strong visuomotor control by iteratively denoising action chunks, but repeated denoising makes real-time deployment computationally demanding. Cache-based methods reduce inference cost by reusing intermediate activations, but existing training-free schedules typically allocate computation uniformly across blocks, ignoring heterogeneous redundancy across blocks and leading to a suboptimal performance-efficiency trade-off. To bridge this gap, we introduce Evolving Cache Schedules (EVO), a training-free acceleration framework that globally schedules cache refreshes via evolutionary search. EVO represents each candidate as a complete schedule over the block-timestep lattice. Thus, redundant transformer computations during iterative denoising can be skipped through cache reuse while preserving closed-loop rollout performance. To make the search practical, EVO introduces redundancy-aware initialization, which seeds the population with promising schedules, and target-conditioned early stopping, which verifies and terminates once a desired performance target is reached. The offline-optimized schedule can be directly plugged into pretrained diffusion policies without retraining. Extensive manipulation benchmarks show that EVO preserves near-full performance while substantially reducing computation, achieving up to 8.05x action-generation speedup and reducing FLOPs from 15.77G to as low as 1.96G. Source code is available at https://github.com/pillom/EVO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。