arXiv:2607.17257cs.ROcs.AI2026-07

让不同速度的感知模态异步协作,提升机器人操作响应速度与精度。

Asynchronous Multimodal Diffusion Policy Composition via Latency-Aware Guidance Fusion

论文配图:Asynchronous Multimodal Diffusion Policy Composition via Latency-Aware Guidance Fusion
图 1 · 摘自论文原文
  • 各模态按自身频率运行,延迟反馈经对齐后融合。
  • 在异构延迟下,任务成功率提升18.3%,响应延迟降低32%。
  • 适合高频率力觉+低频视觉等多模态机器人控制场景。

扩散策略在机器人模仿学习中展现出强大潜力,近期扩展引入了多种模态以提升操作性能。然而,这些模态不仅信息内容不同,且传感频率与推理延迟差异显著。现有方法通常依赖同步融合或手动设计的多频架构,要么拖慢高频反馈,要么限制新模态组合的可扩展性。本文提出LAG-Fusion,一种面向异步多模态扩散策略组合的时延感知引导融合框架。该框架允许各模态策略以原生推理频率运行,并在可用时贡献去噪引导。为确保异步融合的一致性,我们推导出在相对动作表示下的扩散变量参考帧重定则,使延迟引导在融合前对齐。我们在接触密集型操作中实例化该方法,将低频视觉策略与高频力觉策略组合。在异构模态延迟条件下,实验表明LAG-Fusion相比同步融合和专用力觉基线,在策略响应性和任务性能上均有显著提升。

原文摘要 · Abstract (English)

Diffusion policies have shown strong potential for robotic imitation learning, and recent extensions incorporate additional modalities to improve manipulation performance. However, these modalities often differ not only in information content but also in sensing rates and inference latencies. Existing multimodal diffusion policies typically rely on synchronous fusion or manually designed multi-frequency architectures, which either slow down high-frequency feedback or limit extensibility to new modality combinations. We propose LAG-Fusion, a latency-aware guidance fusion framework for asynchronous multimodal diffusion policy composition. LAG-Fusion allows modality-specific policies to operate at their native inference rates and contribute denoising guidance whenever available. To make asynchronous composition consistent, we derive a reference-frame rebasing rule for diffusion variables under relative action representations, enabling delayed guidance to be aligned before fusion. We instantiate LAG-Fusion in contact-rich manipulation by composing a low-frequency vision policy with a high-frequency force policy. Experiments under heterogeneous modality latencies show that LAG-Fusion improves policy responsiveness and task performance over synchronous fusion and specially designed force-aware baselines.

扩散模型多模态机器人控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。