改进机器人生成策略的采样方式,提升推理稳定性与泛化能力
Dense-Jump Flow Matching with Non-Uniform Time Scheduling for Robotic Policies: Mitigating Multi-Step Inference Degradation
- 采用非均匀时间调度训练,强化早期和晚期动作学习
- 推理时用单步跳跃替代多步积分,避免接近终点时性能下降
- 在多种机器人任务中比现有方法提升最高23.7%性能
流匹配已成为机器人生成策略中表现优异的框架;然而我们发现,泛化能力在流轨迹早期即出现并趋于饱和,与文献中的最新发现一致。进一步观察到,推理时增加欧拉积分步数反而普遍降低策略性能。这归因于:(i) 均匀间隔的额外积分步骤过度采样晚期区域,使动作被约束在训练轨迹附近,削弱泛化;(ii) 当积分时间趋近1时,学习到的速度场变得非Lipschitz,导致不稳定。为解决上述问题,我们提出一种新策略:训练阶段使用非均匀时间调度(如U型),强调早期和晚期时间阶段以正则化训练;推理阶段采用密集跳跃积分方案,用单步积分替代跳点后的多步积分,避开接近1处的不稳区域。该策略本质上是高效的单步学习者,仍通过多步积分推动性能提升,在多种机器人任务上相较最优基线最高取得23.7%的性能增益。
原文摘要 · Abstract (English)
Flow matching has emerged as a competitive framework for learning high-quality generative policies in robotics; however, we find that generalisation arises and saturates early along the flow trajectory, in accordance with recent findings in the literature. We further observe that increasing the number of Euler integration steps during inference counter-intuitively and universally degrades policy performance. We attribute this to (i) additional, uniformly spaced integration steps oversample the late-time region, thereby constraining actions towards the training trajectories and reducing generalisation; and (ii) the learned velocity field becoming non-Lipschitz as integration time approaches 1, causing instability. To address these issues, we propose a novel policy that utilises non-uniform time scheduling (e.g., U-shaped) during training, which emphasises both early and late temporal stages to regularise policy training, and a dense-jump integration schedule at inference, which uses a single-step integration to replace the multi-step integration beyond a jump point, to avoid unstable areas around 1. Essentially, our policy is an efficient one-step learner that still pushes forward performance through multi-step integration, yielding up to 23.7% performance gains over state-of-the-art baselines across diverse robotic tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。