让机器人通过对话学习组装新结构，无需事先知道零件规则。
Learning to Assemble Novel Structures with Unfamiliar Parts under Semantic Constraints
- 结合自然语言和视觉信息，从对话中理解新装配规则。
- 用语言说明约束条件比只靠示范更省数据，适应更快。
- 适合需要动态学习新规则的机器人装配场景。
本文提出一种神经符号架构，用于在部署后通过具身对话和任务示范学习组装新颖结构。研究聚焦于代理在运行时遇到训练阶段未提供的语义约束（如哪些零件类型和特征能构成有效结构），且初始未知相关概念的情形。代理需通过与用户的交互获取并利用这些知识进行装配。我们在模拟玩具卡车装配域中验证该方法，从自然语言编码的符号证据和密集视觉观测中学习。实验表明，通过自然语言传达语义约束（如“自卸卡车有翻斗”）比仅依赖任务示范或仅命名零件，能实现更高效的数据在线适应。
原文摘要 · Abstract (English)
This paper describes a neurosymbolic architecture for learning to assemble novel structures using evidence from embodied conversations and task demonstrations. We focus on scenarios where an agent encounters, after deployment, semantic constraints on structures--in other words, constraints as to which part types and features make valid structures--that were not available during training, and where it is initially unaware of the relevant structure and component part concepts. The agent must acquire and exploit such knowledge through user interactions while attempting assembly. We study this setting in a simulated toy truck assembly domain, learning from symbolic evidence encoded in natural language and from dense visual observations. Our experiments show that communicating semantic constraints through natural language (e.g., "dump trucks have a dumper") yields more data-efficient online adaptation than relying only on task demonstrations and/or only naming the parts through natural language.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。