用大模型让普通人也能用自然语言给Misty机器人写代码
AutoMisty: A Multi-Agent LLM Framework for Automated Code Generation in the Misty Social Robot
- 四类智能体协作分解任务并生成代码
- 在真实机器人上测试，效果优于直接用GPT-4o
- 支持自然语言反馈迭代优化，适合非编程用户
社交机器人开放了可定制的通用交互接口，但对无编程经验的用户仍难以使用。本文提出AutoMisty，首个基于大语言模型（LLM）的多智能体协作框架，实现从自然语言指令到可执行Misty机器人代码的自动化生成。AutoMisty包含四个专用智能体模块：任务分解、任务分配、问题求解与结果合成，每个模块均采用双层优化机制，通过自我反思进行迭代优化，并引入人机协同以更好对齐用户偏好。该框架保证推理过程透明，支持用户通过自然语言反馈持续调整任务，实现精准执行。为评估有效性，设计了涵盖四个复杂度层级的基准任务集，并在真实Misty机器人环境中开展实验。大量测试表明，AutoMisty不仅能持续生成高质量代码，还实现了精确的代码控制，显著优于直接使用ChatGPT-4o和ChatGPT-o1的推理方式。所有代码、优化后的API及实验视频将公开发布于 https://wangxiaoshawn.github.io/AutoMisty.html
原文摘要 · Abstract (English)
The social robot's open API allows users to customize open-domain interactions. However, it remains inaccessible to those without programming experience. In this work, we introduce AutoMisty, the first multi-agent collaboration framework powered by large language models (LLMs), to enable the seamless generation of executable Misty robot code from natural language instructions. AutoMisty incorporates four specialized agent modules to manage task decomposition, assignment, problem-solving, and result synthesis. Each agent incorporates a two-layer optimization mechanism, with self-reflection for iterative refinement and human-in-the-loop for better alignment with user preferences. AutoMisty ensures a transparent reasoning process, allowing users to iteratively refine tasks through natural language feedback for precise execution. To evaluate AutoMisty's effectiveness, we designed a benchmark task set spanning four levels of complexity and conducted experiments in a real Misty robot environment. Extensive evaluations demonstrate that AutoMisty not only consistently generates high-quality code but also enables precise code control, significantly outperforming direct reasoning with ChatGPT-4o and ChatGPT-o1. All code, optimized APIs, and experimental videos will be publicly released through the webpage: https://wangxiaoshawn.github.io/AutoMisty.html
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。