arXiv:2508.05298cs.RO2025-08被引 3

让机器人边思考边行动,实时协调多任务执行。

GhostShell: Streaming LLM Function Calls for Concurrent Embodied Programming

  • 通过解析LLM生成流中的函数标记,实现边推理边调用功能。
  • 在33个真实任务中,完成率15/15,比原生调用提升近一倍。
  • 适合需要并发语言与物理动作协同的机器人系统开发。

我们提出GhostShell,一种用于具身系统中流式并发行为编程的新方法。与预设行为结构和计划-执行范式不同,GhostShell通过在LLM流式生成过程中增量调用函数,实现边推理边行动。我们定义了基于XML的函数标记作为函数调用表示,由GhostShell从LLM生成流中解析并映射为可调用函数。多通道调度算法进一步以通道内同步、通道间异步的方式协调这些调用,实现多个机器人组件的串行-并行行为执行。我们在机器人原型CoCo上评估了GhostShell,涵盖9个LLM提供商的33个真实世界任务。在30个具身人机交互(HRI)任务中,使用Claude-Sonnet-4达到0.83的最高定向结构化行为正确率(DSBC);在三个长时程多模态任务中,GPT-4.1获得7.0/10的人工评价分。相比原生LLM函数调用,我们的函数标记方案实现15/15任务完成率,而原方案仅为6/15,尤其在协调语言与物理动作并发方面表现突出。补充材料(含视频)见https://coco-robot.github.io/GhostShell。

原文摘要 · Abstract (English)

We present GhostShell, a novel approach that leverages Large Language Models (LLMs) for streaming and concurrent behavioral programming in embodied systems. In contrast to predefined behavioral structures and plan-then-execute paradigms, GhostShell enables reasoning-while-acting by incrementally invoking functions during LLM streaming generation. We define function tokens as an XML-based function-call representation that GhostShell parses from the LLM generation stream and maps to callable functions. A multi-channel scheduling algorithm further orchestrates these calls with intra-channel synchronous and inter-channel asynchronous dispatch, coordinating sequential-parallel behavior execution across multiple robotic components. We evaluate GhostShell on our robotic prototype CoCo across 33 real-world tasks with LLMs from nine providers. On 30 grounded Human-Robot Interaction (HRI) tasks, our approach achieves the highest Directed Structured Behavior Correctness (DSBC) score of 0.83 with Claude-Sonnet-4, while on three long-horizon multimodal tasks, GPT-4.1 attains a top human evaluation score of 7.0/10. Compared to native LLM function calling, our function token schema achieves a 15/15 task completion rate versus 6/15, particularly in coordinating concurrent linguistic and physical actions. Supplementary materials, including videos, are available at https://coco-robot.github.io/GhostShell.

具身智能流式调用并发控制机器人编程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。