用实时迭代法加速扩散策略推理,无需重训练即可实现实时控制。
Real-Time Iteration Scheme for Diffusion Policy
- 借鉴最优控制的实时迭代思想,用前步解作下一步初始猜测。
- 推理速度提升显著,性能与全步去噪相当,降低计算开销。
- 适合资源密集的大模型,可无缝集成到已有扩散策略中。
扩散策略在机器人操作任务中表现优异,但其长推理时间源于冗长的迭代去噪过程,且需执行一个动作片段后才能进行下一次预测,限制了其在低延迟任务或短周期任务中的应用。尽管近期方法尝试通过蒸馏或替代策略结构加速推理,但通常需要额外训练,对大型机器人模型而言成本较高。本文提出一种受最优控制中实时迭代(RTI)方案启发的新方法,利用前一时步的解作为后续迭代的初始猜测以加速优化。我们还提出一种基于缩放的方法,有效处理机器人操作中的离散动作(如抓取)。该方案无需蒸馏或策略重构,显著降低推理计算开销,可无缝集成至多个预训练的扩散模型,尤其适用于资源需求高的大模型。我们还提供了保证收敛性的理论条件,可用于估计初始去噪步骤。大量仿真实验表明,推理时间大幅减少,整体性能与使用全步去噪的扩散策略相当。
原文摘要 · Abstract (English)
Diffusion Policies have demonstrated impressive performance in robotic manipulation tasks. However, their long inference time, resulting from an extensive iterative denoising process, and the need to execute an action chunk before the next prediction to maintain consistent actions limit their applicability to latency-critical tasks or simple tasks with a short cycle time. While recent methods explored distillation or alternative policy structures to accelerate inference, these often demand additional training, which can be resource-intensive for large robotic models. In this paper, we introduce a novel approach inspired by the Real-Time Iteration (RTI) Scheme, a method from optimal control that accelerates optimization by leveraging solutions from previous time steps as initial guesses for subsequent iterations. We explore the application of this scheme in diffusion inference and propose a scaling-based method to effectively handle discrete actions, such as grasping, in robotic manipulation. The proposed scheme significantly reduces runtime computational costs without the need for distillation or policy redesign. This enables a seamless integration into many pre-trained diffusion-based models, in particular, to resource-demanding large models. We also provide theoretical conditions for the contractivity which could be useful for estimating the initial denoising step. Quantitative results from extensive simulation experiments show a substantial reduction in inference time, with comparable overall performance compared with Diffusion Policy using full-step denoising. Our project page with additional resources is available at: https://rti-dp.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。