让机器人零样本完成复杂抓取任务,靠的是动态智能规划与物理约束结合。
UniManip: General-Purpose Zero-Shot Robotic Manipulation with Agentic Operational Graph
- 用双层智能图结构,把抽象指令和具体动作实时对齐。
- 在未见过的物体和任务上,成功率比顶尖模型高22.5%以上。
- 适合研究通用机器人操作,尤其关注零样本迁移的团队。
实现通用机器人操作需在非结构化环境中无缝连接高层语义意图与底层物理交互。现有方法在零样本泛化上表现不佳:端到端视觉-语言-动作(VLA)模型常缺乏长时任务所需精度,而传统分层规划器在开放世界变化中又显语义僵化。为此,我们提出UniManip,基于双层智能操作图(AOG)框架,统一语义推理与物理具身。通过高层智能层进行任务编排,低层场景层动态表示环境状态,系统持续对齐抽象规划与几何约束,实现鲁棒的零样本执行。不同于静态流程,UniManip以动态智能循环运行:主动从非结构化感知构建以物为中心的场景图,通过安全感知局部规划器参数化无碰撞轨迹,并利用结构化记忆自主诊断与恢复执行失败。大量实验证明,该系统在未见物体与任务上具备强零样本能力,成功率达22.5%和25.0%高于当前最优VLA与分层基线。尤为关键的是,系统可直接零样本迁移至移动操作场景,无需微调或重新配置。开源项目页:https://henryhcliu.github.io/unimanip。
原文摘要 · Abstract (English)
Achieving general-purpose robotic manipulation requires robots to seamlessly bridge high-level semantic intent with low-level physical interaction in unstructured environments. However, existing approaches falter in zero-shot generalization: end-to-end Vision-Language-Action (VLA) models often lack the precision required for long-horizon tasks, while traditional hierarchical planners suffer from semantic rigidity when facing open-world variations. To address this, we present UniManip, a framework grounded in a Bi-level Agentic Operational Graph (AOG) that unifies semantic reasoning and physical grounding. By coupling a high-level Agentic Layer for task orchestration with a low-level Scene Layer for dynamic state representation, the system continuously aligns abstract planning with geometric constraints, enabling robust zero-shot execution. Unlike static pipelines, UniManip operates as a dynamic agentic loop: it actively instantiates object-centric scene graphs from unstructured perception, parameterizes these representations into collision-free trajectories via a safety-aware local planner, and exploits structured memory to autonomously diagnose and recover from execution failures. Extensive experiments validate the system's robust zero-shot capability on unseen objects and tasks, demonstrating a 22.5% and 25.0% higher success rate compared to state-of-the-art VLA and hierarchical baselines, respectively. Notably, the system enables direct zero-shot transfer from fixed-base setups to mobile manipulation without fine-tuning or reconfiguration. Our open-source project page can be found at https://henryhcliu.github.io/unimanip.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。