arXiv:2603.28156cs.RO2026-03

让AI机器人出错时自动求助人类,减少操作负担。

Reducing Mental Workload through On-Demand Human Assistance for Physical Action Failures in LLM-based Multi-Robot Coordination

  • AI机器人执行中出错时,自动请求远程人类协助恢复
  • 实测显示任务完成量比纯自主模式提升显著,易捡物品接近全程遥控效果
  • 减轻操作员心理负担,尤其降低重复性体力消耗

基于大语言模型(LLMs)的多机器人协同正日益受到关注,因LLM可将自然语言指令直接转化为机器人动作计划,实现任务分解与高层规划。然而,物理执行失败后的恢复仍面临挑战,常导致任务停滞于重复无效动作。尽管已有结合混合现实的远程机器人操作框架,但针对多机器人环境中物理失败的远程纠错仍缺乏系统方案。本文提出REPAIR(Robot Execution with Planned And Interactive Recovery)——一种将远程错误修复整合进LLM驱动多机器人规划的人机协同框架。该方法下,机器人自主执行任务;当发生无法自愈的故障时,由LLM主动向操作员请求帮助,实现通过远程干预维持任务连续性。在真实环境中的多机器人垃圾清理任务测试表明,相较于完全自主方法,REPAIR显著提升了任务进度(时间限制内清理物品数量)。对于易拾取物品,其表现接近全远程控制水平。结果还提示,操作员的心理负荷在物理需求和努力程度上存在差异。

原文摘要 · Abstract (English)

Multi-robot coordination based on large language models (LLMs) has attracted growing attention, since LLMs enable the direct translation of natural language instructions into robot action plans by decomposing tasks and generating high-level plans. However, recovering from physical execution failures remains difficult, and tasks often stagnate due to the repetition of the same unsuccessful actions. While frameworks for remote robot operation using Mixed Reality were proposed, there have been few attempts to implement remote error resolution specifically for physical failures in multi-robot environments. In this study, we propose REPAIR (Robot Execution with Planned And Interactive Recovery), a human-in-the-loop framework that integrates remote error resolution into LLM-based multi-robot planning. In this method, robots execute tasks autonomously; however, when an irrecoverable failure occurs, the LLM requests assistance from an operator, enabling task continuity through remote intervention. Evaluations using a multi-robot trash collection task in a real-world environment confirmed that REPAIR significantly improves task progress (the number of items cleared within a time limit) compared to fully autonomous methods. Furthermore, for easily collectable items, it achieved task progress equivalent to full remote control. The results also suggested that the mental workload on the operator may differ in terms of physical demand and effort. The project website is https://emergentsystemlabstudent.github.io/REPAIR/.

人机协同多机器人心理负荷故障恢复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。