arXiv:2512.07212cs.AIcs.LG2025-12中稿 · ICML被引 2

让机器人从看到的场景直接生成动作,提升控制精度与可靠性。

Sample from What You See: Visuomotor Policy Learning via Diffusion Bridge with Observation-Embedded Stochastic Differential Equation

  • 将观察信息融入扩散过程动态,而非仅作条件输入
  • 在52个仿真任务和5个真实任务中均超越现有方法
  • 适合需要高精度视觉-动作对齐的机器人控制场景

基于扩散模型的模仿学习已推动机器人控制发展,能捕捉多模态动作分布。但现有方法通常仅将观测作为去噪网络的高层条件,未将其融入扩散过程本身的随机动力学。因此采样必须从随机噪声开始,削弱了感知与控制的耦合,常导致性能不佳。本文提出BridgePolicy,一种通过扩散桥框架将观测直接嵌入随机动力学的生成式视觉-运动策略。通过构建受观测引导的轨迹,使采样可从丰富且信息量大的先验开始,显著提升控制精度与可靠性。关键挑战在于扩散桥通常要求分布维度匹配,而机器人观测具有异质性,无法自然对齐动作。为此,我们引入语义对齐器,统一视觉与状态输入,并将观测与动作表示对齐,使扩散桥适用于异构机器人数据。在三个基准上的52个仿真任务及5个真实世界任务中,BridgePolicy持续优于最先进生成策略。代码已开源。

原文摘要 · Abstract (English)

Imitation learning with diffusion models has advanced robotic control by capturing the multi-modal action distributions. However, existing methods typically treat observations only as high-level conditions to the denoising network, rather than integrating them into the stochastic dynamics of the diffusion process itself. As a result, the sampling is forced to begin from random noise, weakening the coupling between perception and control and often yielding suboptimal performance. We propose BridgePolicy, a generative visuomotor policy that directly integrates observations into the stochastic dynamics via a diffusion-bridge formulation. By constructing an observation-informed trajectory, BridgePolicy enables sampling to start from a rich and informative prior rather than random noise, substantially improving precision and reliability in control. A key difficulty is that diffusion bridge normally connects distributions of matched dimensionality, while robotic observations are heterogeneous and not naturally aligned with actions. To overcome this, we introduce a semantic aligner to unify the visual and state inputs and align the observations with action representations, making diffusion bridge applicable to heterogeneous robot data. Extensive experiments across 52 simulation tasks on three benchmarks and 5 real-world tasks demonstrate that BridgePolicy consistently outperforms state-of-the-art generative policies. Our code is available at https://jianghcsr.github.io/BridgePolicy_page/.

扩散模型机器人控制视觉动作对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。