让大模型函数调用异步执行，不改模型也能提速
Concurrency without Model Changes: Future-based Asynchronous Function Calling for LLMs
- 在不修改模型和函数的前提下，实现解码与调用并行
- 测试显示任务完成时间显著降低，准确率保持不变
- 揭示大模型能天然处理未完成的执行结果符号
函数调用是现代大模型智能体的核心能力，但传统同步执行方式会阻塞模型解码，导致端到端延迟增加。本文提出 AsyncFC，一个纯执行层框架，将模型解码与函数执行解耦，实现解码与调用重叠以及函数间的并行执行（当依赖允许时）。该框架无需微调或修改现有模型和函数实现，兼容标准同步调用协议。在标准函数调用基准和适配的软件工程基准上，AsyncFC 显著缩短任务完成时间，同时保持任务准确率。此外，实验表明大模型具备原生推理符号化未来（即未完成执行结果）的能力，支持异步模型-工具交互范式。
原文摘要 · Abstract (English)
Function calling, also known as tool use, is a core capability of modern LLM agents but is typically constrained by synchronous execution semantics. Under these semantics, LLM decoding is blocked until each function call completes, resulting in increasing end-to-end latency. In this work, we introduce AsyncFC, a pure execution-layer framework that decouples LLM decoding from function execution, enabling overlap between model decoding and function execution as well as inter-function parallelism when dependencies permit. AsyncFC layers over existing models and unmodified function implementations, requiring no fine-tuning or changes to the standard synchronous function-calling protocol. Across standard function-calling benchmarks and adapted software engineering benchmarks, AsyncFC significantly reduces end-to-end task completion time while preserving task accuracy. Furthermore, these results reveal that LLMs possess a native capability to reason over symbolic futures that represent unresolved execution results, enabling an asynchronous paradigm for model-tool interaction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。