arXiv:2602.23024cs.RO2026-02

让机器人移动抓取更智能:动态分配注意力,解耦控制动作

InCoM: Intent-Driven Perception and Structured Coordination for Mobile Manipulation

  • 根据任务意图动态调整视觉感知重点,适应视角变化
  • 在三个真实场景中成功率提升超23%,超越现有方法
  • 适合需要移动与操作协同的机器人研究者和工程师

移动操作是通用机器人代理的核心能力,要求同时协调移动底盘与机械臂,并在视角动态变化下保持鲁棒感知。然而,现有方法存在两大挑战:底盘与机械臂动作强耦合,增加控制优化难度;视角变化时感知注意力分配不佳。本文提出InCoM框架,通过推断潜在运动意图,动态重加权多尺度感知特征,实现阶段自适应的感知注意力分配。为支持跨模态鲁棒感知,进一步引入几何-语义结构对齐机制,增强多模态对应关系。控制方面,设计解耦的协同流匹配动作解码器,显式建模底盘-机械臂协同动作生成,缓解控制耦合带来的优化难题。实验表明,InCoM在无特权信息条件下,于三个ManiSkill-HAB场景中分别取得28.2%、26.1%、23.6%的成功率提升。其有效性在真实世界移动操作任务中也得到一致验证,持续优于现有基线。

原文摘要 · Abstract (English)

Mobile manipulation is a fundamental capability for general-purpose robotic agents, requiring both coordinated control of the mobile base and manipulator and robust perception under dynamically changing viewpoints. However, existing approaches face two key challenges: strong coupling between base and arm actions complicates control optimization, and perceptual attention is often poorly allocated as viewpoints shift during mobile manipulation. We propose InCoM, an intent-driven perception and structured coordination framework for mobile manipulation. InCoM infers latent motion intent to dynamically reweight multi-scale perceptual features, enabling stage-adaptive allocation of perceptual attention. To support robust cross-modal perception, InCoM further incorporates a geometric-semantic structured alignment mechanism that enhances multimodal correspondence. On the control side, we design a decoupled coordinated flow matching action decoder that explicitly models coordinated base-arm action generation, alleviating optimization difficulties caused by control coupling. Experimental results demonstrate that InCoM significantly outperforms state-of-the-art methods, achieving success rate gains of 28.2%, 26.1%, and 23.6% across three ManiSkill-HAB scenarios without privileged information. Furthermore, its effectiveness is consistently validated in real-world mobile manipulation tasks, where InCoM maintains a superior success rate over existing baselines.

移动操作感知对齐动作解耦机器人控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。