arXiv:2512.18337cs.CL2025-12被引 6

提出统一框架,显著提升大模型智能体的推理效率与长期任务稳定性。

Towards Efficient Agents: A Co-Design of Inference Architecture and System

  • 分层双模型动态分配角色,优化大小模型协同
  • 减少50%无效词元消耗,速度提升1.8至2.5倍
  • 适合长周期推理任务的高效智能体系统设计

基于大语言模型的智能体在自主多轮推理和工具增强决策方面展现出巨大潜力,但其实际部署受限于推理循环、上下文增长和异构工具交互带来的系统性延迟。本文提出AgentInfer,一个端到端智能体加速的统一框架,融合推理优化与架构设计。通过四个协同组件:AgentCollab(分层双模型动态角色分配)、AgentSched(缓存感知混合调度器)、AgentSAM(基于后缀自动机的推测解码,复用多会话语义记忆)以及AgentCompress(异步语义压缩机制,不中断推理),构建自演化引擎,在长时序任务中维持高效与认知稳定。在BrowseComp-zh和DeepDiver基准上实验表明,协同应用该框架使无效词元消耗降低超50%,整体速度提升1.8–2.5倍,且准确率保持不变。结果表明,以任务完成效率为核心优化目标,而非单纯每词元吞吐量,是构建可扩展、高效且自我演进智能系统的关键。

原文摘要 · Abstract (English)

The rapid development of large language model (LLM)-based agents has unlocked new possibilities for autonomous multi-turn reasoning and tool-augmented decision-making. However, their real-world deployment is hindered by severe inefficiencies that arise not from isolated model inference, but from the systemic latency accumulated across reasoning loops, context growth, and heterogeneous tool interactions. This paper presents AgentInfer, a unified framework for end-to-end agent acceleration that bridges inference optimization and architectural design. We decompose the problem into four synergistic components: AgentCollab, a hierarchical dual-model reasoning framework that balances large- and small-model usage through dynamic role assignment; AgentSched, a cache-aware hybrid scheduler that minimizes latency under heterogeneous request patterns; AgentSAM, a suffix-automaton-based speculative decoding method that reuses multi-session semantic memory to achieve low-overhead inference acceleration; and AgentCompress, a semantic compression mechanism that asynchronously distills and reorganizes agent memory without disrupting ongoing reasoning. Together, these modules form a Self-Evolution Engine capable of sustaining efficiency and cognitive stability throughout long-horizon reasoning tasks. Experiments on the BrowseComp-zh and DeepDiver benchmarks demonstrate that through the synergistic collaboration of these methods, AgentInfer reduces ineffective token consumption by over 50%, achieving an overall 1.8-2.5 times speedup with preserved accuracy. These results underscore that optimizing for agentic task completion-rather than merely per-token throughput-is the key to building scalable, efficient, and self-improving intelligent systems.

智能体推理优化系统设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。