让机器人通过大模型+知识图谱+人类反馈，快速学会新任务
AdaptBot: Combining LLM with Knowledge Graphs and Human Input for Generic-to-Specific Task Decomposition and Knowledge Refinement
- 结合大模型的泛化能力与知识图谱的领域知识，生成可执行的任务分解
- 在模拟烹饪和清洁任务中，性能显著优于仅用大模型的方法
- 适合需要快速适应新任务的具身智能体研发人员参考
一个协助人类的具身智能体常需完成新任务，但缺乏足够时间或标注数据进行训练。基于广泛领域知识训练的大语言模型（LLM）可预测完成任务所需的抽象动作序列，但受任务、智能体或领域特定约束，可能无法直接执行。本文提出框架，融合LLM的通用预测与知识图谱（KG）中的先验领域知识，使智能体能快速适应新任务，并在必要时主动请求并利用人类输入来优化现有知识。在模拟烹饪和清洁任务中的实验表明，LLM、KG与人类输入的协同作用显著提升了性能，优于仅使用LLM的方法。
原文摘要 · Abstract (English)
An embodied agent assisting humans is often asked to complete new tasks, and there may not be sufficient time or labeled examples to train the agent to perform these new tasks. Large Language Models (LLMs) trained on considerable knowledge across many domains can be used to predict a sequence of abstract actions for completing such tasks, although the agent may not be able to execute this sequence due to task-, agent-, or domain-specific constraints. Our framework addresses these challenges by leveraging the generic predictions provided by LLM and the prior domain knowledge encoded in a Knowledge Graph (KG), enabling an agent to quickly adapt to new tasks. The robot also solicits and uses human input as needed to refine its existing knowledge. Based on experimental evaluation in the context of cooking and cleaning tasks in simulation domains, we demonstrate that the interplay between LLM, KG, and human input leads to substantial performance gains compared with just using the LLM. Project website§: https://sssshivvvv.github.io/adaptbot/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。