机器人通过对话持续学习，避免重复错误，提升适应能力。
iLearnRobot: An Interactive Learning-Based Multi-Modal Robot with Continuous Improvement
- 基于多模态大模型，从用户自然对话中学习
- 采用问答链澄清意图，双模态检索防止重复错误
- 适合需要长期交互的智能服务机器人场景
机器人在部署后仍需持续改进，因其可能遭遇前所未见的新场景。本文提出一种基于多模态大语言模型（MLLM）的交互式学习机器人系统，能从非专家用户的自然对话中学习。系统引入问答链机制，在回答前明确用户意图，并设计双模态检索模块，利用交互记录避免重复错误，确保更新前的流畅体验，区别于当前主流的MLLM驱动机器人系统。实验表明，该方法在定量与定性层面均有效提升了性能，为机器人持续适应复杂环境提供了新路径。
原文摘要 · Abstract (English)
It is crucial that robots' performance can be improved after deployment, as they are inherently likely to encounter novel scenarios never seen before. This paper presents an innovative solution: an interactive learning-based robot system powered by a Multi-modal Large Language Model(MLLM). A key feature of our system is its ability to learn from natural dialogues with non-expert users. We also propose chain of question to clarify the exact intent of the question before providing an answer and dual-modality retrieval modules to leverage these interaction events to avoid repeating same mistakes, ensuring a seamless user experience before model updates, which is in contrast to current mainstream MLLM-based robotic systems. Our system marks a novel approach in robotics by integrating interactive learning, paving the way for superior adaptability and performance in diverse environments. We demonstrate the effectiveness and improvement of our method through experiments, both quantitively and qualitatively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。