让机器人用自然语言协作求助，自动评估影响并选最优帮手。
Seeing, Saying, Solving: An LLM-to-TL Framework for Cooperative Robots
- 用视觉语言模型识冲突，大模型生成自然语言求助
- 帮手用逻辑语法确保指令正确，求解优化任务影响
- 综合考虑多请求可减少系统总耗时，优于就近选择
随着仓储等领域机器人部署增多，异构机器人团队需应对突发冲突。本文提出一种去中心化协作框架：机器人通过视觉语言模型（VLM）检测冲突，若需帮助，则由大语言模型（LLM）生成自然语言求助信息并广播。潜在帮手机器人基于信号时序逻辑（STL）与巴科斯-诺尔范式（BNF）语法，将自然语言请求转化为语法合法的STL表达式，并以混合整数线性规划（MILP）求解其任务影响。请求方根据各帮手的影响评估，选择最优协助者。实验表明，综合比较多个帮助方案可显著降低系统整体时间开销，优于仅选择最近机器人的启发式策略。
原文摘要 · Abstract (English)
Increased robot deployment, such as in warehousing, has revealed a need for seamless collaboration among heterogeneous robot teams to resolve unforeseen conflicts. To address this challenge, we propose a novel, decentralized framework for robots to request and provide help. The framework begins with robots detecting conflicts using a Vision Language Model (VLM), then reasoning over whether help is needed. If so, it crafts and broadcasts a natural language (NL) help request using a Large Language Model (LLM). Potential helper robots reason over the request and offer help (if able), along with information about impact to their current tasks. Helper reasoning is implemented via an LLM grounded in Signal Temporal Logic (STL) using a Backus-Naur Form (BNF) grammar to guarantee syntactically valid NL-to-STL translations, which are then solved as a Mixed Integer Linear Program (MILP). Finally, the requester robot chooses a helper by reasoning over impact on the overall system. We evaluate our system via experiments considering different strategies for choosing a helper, and find that a requester robot can minimize overall time impact on the system by considering multiple help offers versus simple heuristics (e.g., selecting the nearest robot to help).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。