让大模型函数调用不再阻塞，实现并行执行，提升效率。
Asynchronous LLM Function Calling
- 采用异步中断机制，允许模型在等待调用返回时继续推理。
- 在伯克利基准测试中，任务完成延迟降低1.6倍至5.4倍。
- 支持人机或模型间新型交互，适合高并发应用。
大型语言模型（LLMs）通过函数调用与外部工具和数据源交互。然而，当前方法本质上是同步的，每次调用都会阻塞模型推理，限制了操作效率和函数并发执行能力。本文提出AsyncLM，一种支持异步函数调用的系统。AsyncLM通过引入中断机制，在函数调用返回时异步通知正在运行的LLM，从而实现函数生成与执行的并发。我们设计了上下文感知的函数调用与中断协议，提出适配中断语义的微调策略，并高效集成到推理流程中。在伯克利函数调用排行榜（BFCL）的一系列基准任务上，AsyncLM将端到端任务完成延迟降低了1.6倍至5.4倍。此外，我们探讨了中断机制如何扩展以支持新型人-模型或模型-模型交互。
原文摘要 · Abstract (English)
Large language models (LLMs) use function calls to interface with external tools and data source. However, the current approach to LLM function calling is inherently synchronous, where each call blocks LLM inference, limiting LLM operation and concurrent function execution. In this work, we propose AsyncLM, a system for asynchronous LLM function calling. AsyncLM improves LLM's operational efficiency by enabling LLMs to generate and execute function calls concurrently. Instead of waiting for each call's completion, AsyncLM introduces an interrupt mechanism to asynchronously notify the LLM in-flight when function calls return. We design an in-context protocol for function calls and interrupts, provide fine-tuning strategy to adapt LLMs to the interrupt semantics, and implement these mechanisms efficiently on LLM inference process. We demonstrate that AsyncLM can reduce end-to-end task completion latency from 1.6x-5.4x compared to synchronous function calling on a set of benchmark tasks in the Berkeley function calling leaderboard (BFCL). Furthermore, we discuss how interrupt mechanisms can be extended to enable novel human-LLM or LLM-LLM interactions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。