构建可解析的结构化记忆,让机器人更好记住物体与操作关系。
Analytic Concept-Centric Memory for Agentic Embodied Manipulation

- 用语义部件、参数模板等结构化表示物体和动作状态
- 实现精准的物体重识别与跨对象技能复用,任务完成率提升23%
- 适合需要长期记忆与复杂操作的具身智能任务
长时程具身操作要求智能体持续记忆物体、追踪场景变化并复用交互经验。现有记忆多为非结构化历史或嵌入式记录,难以检索与操作相关的物体部件、物理状态、动作效果及可执行技能。本文提出一种分析性概念中心记忆框架,将经验围绕结构化分析概念组织,物体由语义部件、参数模板、接地姿态、可操作性及操作状态表示。该框架进一步通过过渡记忆连接物体与场景记忆,通过技能记忆支持模板与策略驱动的执行。运行时,智能体进行结构化的粗粒度到细粒度检索,实现状态一致推理与技能复用。在依赖记忆的操作、可动物体泛化、真实世界记忆评估及消融实验中,本方法在任务完成率、检索准确率、物体重识别率与跨物体技能泛化方面均优于非结构化和嵌入式记忆基线。
原文摘要 · Abstract (English)
Long-horizon embodied manipulation requires agents to remember persistent objects, track changing scene states, and reuse prior interaction knowledge. However, existing agent memories are often stored as unstructured histories or embedding-based records, making it difficult to retrieve manipulation-relevant object parts, physical states, action effects, and executable skills. We propose an analytic concept-centric memory framework for agentic embodied manipulation. Our memory organizes experience around structured analytic concepts, where objects are represented by semantic parts, parametric templates, grounded poses, affordances, and manipulation states. It further connects object and scene memories with transition memory for action-induced state changes and skill memory for template-grounded and policy-grounded execution. At runtime, the agent performs structured coarse-to-fine retrieval to identify relevant objects, states, transitions, and skills, supporting state-consistent reasoning and skill reuse. Experiments on memory-dependent manipulation, articulated-object generalization, real-world memory evaluation, and ablations show that our approach improves task completion, retrieval accuracy, object re-identification, and cross-object skill generalization over unstructured and embedding-based memory baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。