用流匹配方法让机器人在新任务中更稳定地模仿动作。
ContextFlow: In-Context Flow Matching for Robot Manipulation

- 用流匹配模型直接预测连续动作,避免误差累积。
- 在未见过的任务配置上成功率比现有方法高35个百分点。
- 无需微调即可在真实机器人上实现40%的泛化成功率。
尽管在视觉和语言领域表现优异,将上下文学习应用于机器人仍具挑战性。现有自回归式上下文模仿方法对连续动作进行离散化,通过逐词预测加剧早期误差积累,限制了在未见任务配置上的泛化能力。而流匹配策略虽可用于连续机器人控制并缓解误差累积,但其在上下文模仿学习框架中的应用尚未充分探索。为此,本文提出ContextFlow,一种条件流匹配模型,用于学习上下文模仿中的连续动作分布。该模型基于示范与观测条件生成动作,能从噪声动作分布中鲁棒生成。为更好编码多模态上下文示范,我们引入类Perceiver的多模态上下文压缩器,将视觉、本体感知和动作序列压缩为紧凑且任务相关的潜在表示。在LIBERO数据集上,ContextFlow在未见任务配置上的平均成功率比ICRT高出35个百分点,且无需任何微调即达到任务专属微调模型π₀的性能。在真实机器人上,其成功泛化至单臂与双臂任务的未见配置,新笔帽开启任务的成功率达40%。
原文摘要 · Abstract (English)
Although highly effective in vision and language domains, applying in-context learning to robotics remains challenging. Existing autoregressive in-context imitation methods discretize continuous actions and exacerbate the accumulation of early prediction errors through next-token prediction, limiting their generalization on unseen task configurations. Meanwhile, flow-matching policies have been explored for continuous robot control and can help mitigate compounding errors; however, in-context imitation learning within a flow-matching framework remains underexplored. To address these limitations, we introduce ContextFlow, a conditional flow-matching model that learns continuous action distributions for in-context imitation learning. ContextFlow conditions flow-based action prediction on demonstrations and observations, enabling robust generation from noisy action distributions. To better encode multimodal in-context demonstrations, we adapt perceiver-style multimodal context compressors that distill visual, proprioceptive, and action sequences into compact, task-relevant latent representations. On LIBERO, ContextFlow outperforms ICRT by 35 percentage points in average success rate on unseen task configurations, while matching the performance of the task-specific fine-tuned VLA model $\pi_0$ without any fine-tuning on unseen tasks. On real robots, it generalizes to unseen configurations of both single-arm and bimanual tasks, achieving 40% success on a new pen-uncapping configuration. Project Page: https://dingjiansw101.github.io/contextflow-page/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。