arXiv:2505.15659cs.ROcs.LG2025-05被引 87

让机器人通过预测未来视觉特征来提前规划动作,提升长程决策能力。

FLARE: Robot Learning with Implicit World Modeling

  • 用扩散模型对齐未来视觉特征,实现对未来状态的隐式建模。
  • 在双臂与人形机器人任务中性能领先,最高提升26%。
  • 仅需少量修改即可支持无动作标签的人类视角视频训练,泛化更强。

我们提出一种新框架FLARE(未来隐式世界表征对齐),将预测性隐式世界建模融入机器人策略学习。通过将扩散变换器的特征与未来观测的隐向量对齐,FLARE使扩散变换器策略能够预判未来观测的隐表示,从而在生成动作时推理长期后果。该方法极为轻量,仅需在标准视觉-语言-动作(VLA)模型中增加少量标记即可,却带来显著性能提升。在涵盖单臂与人形机器人桌面操作的两个挑战性多任务仿真模仿学习基准上,FLARE达到当前最优表现,较先前基线最高提升26%。此外,FLARE可实现无需动作标签的人类第一人称视频示范联合训练,仅用一次机器人演示即可显著提升对未知几何新物体的泛化能力。结果表明,FLARE是一种通用且可扩展的隐式世界建模与高频机器人控制结合方法。

原文摘要 · Abstract (English)

We introduce $\textbf{F}$uture $\textbf{LA}$tent $\textbf{RE}$presentation Alignment ($\textbf{FLARE}$), a novel framework that integrates predictive latent world modeling into robot policy learning. By aligning features from a diffusion transformer with latent embeddings of future observations, $\textbf{FLARE}$ enables a diffusion transformer policy to anticipate latent representations of future observations, allowing it to reason about long-term consequences while generating actions. Remarkably lightweight, $\textbf{FLARE}$ requires only minimal architectural modifications -- adding a few tokens to standard vision-language-action (VLA) models -- yet delivers substantial performance gains. Across two challenging multitask simulation imitation learning benchmarks spanning single-arm and humanoid tabletop manipulation, $\textbf{FLARE}$ achieves state-of-the-art performance, outperforming prior policy learning baselines by up to 26%. Moreover, $\textbf{FLARE}$ unlocks the ability to co-train with human egocentric video demonstrations without action labels, significantly boosting policy generalization to a novel object with unseen geometry with as few as a single robot demonstration. Our results establish $\textbf{FLARE}$ as a general and scalable approach for combining implicit world modeling with high-frequency robotic control.

机器人学习扩散模型隐式建模模仿学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。