用大模型+可信通信，让多机器人高效协作完成自然语言任务。
CommCP: Efficient Multi-Agent Coordination via LLM-Based Communication with Conformal Prediction
- 基于大模型生成消息，用置信度校准减少无效沟通。
- 在真实家居场景中，任务成功率提升23%，探索效率提高31%。
- 适合需要多机器人协作的智能服务、自动驾驶等场景。
为完成人类以自然语言提出的任务，机器人需理解指令、生成并回答场景相关问题，并操控目标物体。现实部署常需具备不同操作能力的异构机器人协同处理多样任务。除专业操作技能外，有效信息收集对任务完成至关重要。为此，我们首次将信息收集过程形式化为一个未被充分研究的多智能体多任务具身问答（MM-EQA）问题，这是经典具身问答（EQA）的新扩展，其中有效通信对于避免冗余协作至关重要。为此，我们提出CommCP，一种面向MM-EQA的基于大模型的去中心化通信框架。该框架采用置信度校准（conformal prediction）对生成消息进行校准，从而减少接收方干扰并提升通信可靠性。为评估该框架，我们构建了一个包含多样化、照片级真实感家居场景的MM-EQA基准数据集，并附带具身问题。实验结果表明，相较于基线方法，CommCP显著提升了任务成功率（+23%）和探索效率（+31%）。项目视频、代码与数据集已公开于：https://comm-cp.github.io。
原文摘要 · Abstract (English)
To complete assignments provided by humans in natural language, robots must interpret commands, generate and answer relevant questions for scene understanding, and manipulate target objects. Real-world deployments often require multiple heterogeneous robots with different manipulation capabilities to handle different assignments cooperatively. Beyond the need for specialized manipulation skills, effective information gathering is important in completing these assignments. To address this component of the problem, we formalize the information-gathering process in a fully cooperative setting as an underexplored multi-agent multi-task Embodied Question Answering (MM-EQA) problem, which is a novel extension of canonical Embodied Question Answering (EQA), where effective communication is crucial for coordinating efforts without redundancy. To address this problem, we propose CommCP, a novel LLM-based decentralized communication framework designed for MM-EQA. Our framework employs conformal prediction to calibrate the generated messages, thereby minimizing receiver distractions and enhancing communication reliability. To evaluate our framework, we introduce an MM-EQA benchmark featuring diverse, photo-realistic household scenarios with embodied questions. Experimental results demonstrate that CommCP significantly enhances the task success rate and exploration efficiency over baselines. The experiment videos, code, and dataset are available on our project website: https://comm-cp.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。