让扩散策略训练更快更稳,关键在调整条件输入的分布方式。
Conditioning Matters: Training Diffusion Policies is Faster Than You Think
- 通过让条件输入决定生成起点分布,增强条件感知能力。
- 训练收敛速度提升,用更少梯度步数达到大模型同等性能。
- 方法轻量通用,适合各类扩散策略架构,实测效果显著。
扩散策略已成为视觉-语言-动作(VLA)模型的主流范式,虽具强大机器人控制能力,但训练效率仍不理想。本文识别出条件扩散策略训练中的根本问题:当生成条件难以区分时,训练目标退化为建模动作的边缘分布,称为损失坍塌。为此提出Cocos,一种简单通用的解决方案——在条件流匹配中使源分布依赖于条件。通过将源分布锚定在条件输入提取的语义附近,Cocos强化了条件融合,防止损失坍塌。我们提供了理论依据及大量仿真与真实世界基准的实证结果。该方法实现更快收敛与更高成功率,仅用更少梯度步数和参数即达到大规模预训练VLA性能。Cocos轻量易实现,兼容多种策略架构,可作为扩散策略训练的通用改进方案。
原文摘要 · Abstract (English)
Diffusion policies have emerged as a mainstream paradigm for building vision-language-action (VLA) models. Although they demonstrate strong robot control capabilities, their training efficiency remains suboptimal. In this work, we identify a fundamental challenge in conditional diffusion policy training: when generative conditions are hard to distinguish, the training objective degenerates into modeling the marginal action distribution, a phenomenon we term loss collapse. To overcome this, we propose Cocos, a simple yet general solution that modifies the source distribution in the conditional flow matching to be condition-dependent. By anchoring the source distribution around semantics extracted from condition inputs, Cocos encourages stronger condition integration and prevents the loss collapse. We provide theoretical justification and extensive empirical results across simulation and real-world benchmarks. Our method achieves faster convergence and higher success rates than existing approaches, matching the performance of large-scale pre-trained VLAs using significantly fewer gradient steps and parameters. Cocos is lightweight, easy to implement, and compatible with diverse policy architectures, offering a general-purpose improvement to diffusion policy training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。