arXiv:2602.13665cs.AI2026-02KDD被引 1

用轻量级流水线+动态模板,让大模型调用函数快8倍还更准

HyFunc: Accelerating LLM-based Function Calls for Agentic AI through Hybrid-Model Cascade and Dynamic Templating

  • 大模型只提炼意图符号,小模型负责选函数和生成调用
  • 推理延迟0.828秒,性能达80.1%,比同类模型更快更准
  • 适合需要低延迟函数调用的智能体系统开发者

尽管智能体系统依赖大模型将用户意图转化为结构化函数调用,但该过程存在计算冗余,导致高推理延迟,阻碍实时应用。本文识别并解决三大冗余:(1)每次请求重复处理大量函数描述;(2)使用大型慢模型生成通常可预测的完整标记序列;(3)重复生成固定模板参数语法。我们提出HyFunc框架,系统性消除这些效率瓶颈。HyFunc采用混合模型级联,大模型将用户意图提炼为单一“软标记”,该标记引导轻量检索器选择相关函数,并指导小型前缀微调模型生成最终调用,避免大模型重复上下文处理与全序列生成。为消除语法冗余,提出“动态模板”技术,在扩展vLLM引擎中实时注入模板语法。为规避泛化局限,我们在未见过的BFCL基准数据集上评估。实验表明,HyFunc在效率与性能间取得良好平衡:推理延迟为0.828秒,优于所有基线模型;性能达80.1%,超过同等参数规模模型。结果表明,HyFunc为智能体系统提供了更高效的函数调用范式。代码已公开于https://github.com/MrBlankness/HyFunc。

原文摘要 · Abstract (English)

While agentic AI systems rely on LLMs to translate user intent into structured function calls, this process is fraught with computational redundancy, leading to high inference latency that hinders real-time applications. This paper identifies and addresses three key redundancies: (1) the redundant processing of a large library of function descriptions for every request; (2) the redundant use of a large, slow model to generate an entire, often predictable, token sequence; and (3) the redundant generation of fixed, boilerplate parameter syntax. We introduce HyFunc, a novel framework that systematically eliminates these inefficiencies. HyFunc employs a hybrid-model cascade where a large model distills user intent into a single "soft token." This token guides a lightweight retriever to select relevant functions and directs a smaller, prefix-tuned model to generate the final call, thus avoiding redundant context processing and full-sequence generation by the large model. To eliminate syntactic redundancy, our "dynamic templating" technique injects boilerplate parameter syntax on-the-fly within an extended vLLM engine. To avoid potential limitations in generalization, we evaluate HyFunc on an unseen benchmark dataset, BFCL. Experimental results demonstrate that HyFunc achieves an excellent balance between efficiency and performance. It achieves an inference latency of 0.828 seconds, outperforming all baseline models, and reaches a performance of 80.1%, surpassing all models with a comparable parameter scale. These results suggest that HyFunc offers a more efficient paradigm for agentic AI. Our code is publicly available at https://github.com/MrBlankness/HyFunc.

智能体系统函数调用推理加速vLLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。