为智能体工作流设计高效推理系统，提升多步骤调用的复用效率。
Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective
- 将智能体工作流视为查询计划，把LLM调用当作可优化的操作符。
- 通过预缓存和缓存感知调度，实现跨提示词和中间状态的复用，最高提速1.56倍。
- 适合需要高效处理复杂多步任务的AI系统开发者与部署工程师。
智能体工作流由一系列相互依赖的大语言模型（LLM）调用组成，已成为现代AI系统的主要负载。这些工作流因推测性与并行探索导致大量重复的提示词和中间结果，现有LLM服务系统（如vLLM）仅优化单个推理调用，忽略跨调用依赖，造成显著效率损失。本文从数据系统视角重新思考LLM与智能体服务，提出Helium框架，将智能体工作负载建模为查询计划，将LLM调用视为第一类操作符。Helium集成主动缓存与缓存感知调度，最大化提示词、键值状态和工作流间的复用。通过这些技术，将经典查询优化原理引入LLM服务，在多种工作负载上实现较当前最优智能体服务系统最高1.56倍的加速。结果表明，端到端的工作流优化对构建可扩展、高效的基于LLM的智能体至关重要。
原文摘要 · Abstract (English)
Agentic workflows are composed of sequences of interdependent Large Language Model (LLM) calls, and they have become a dominant workload in modern AI systems. These workflows exhibit extensive redundancy from overlapping prompts and intermediate results due to speculative and parallel exploration. Existing LLM serving systems, such as vLLM, focus on optimizing individual inference calls and overlook cross-call dependencies, leading to significant inefficiencies. This paper rethinks LLM and agent serving from a data systems perspective and introduces Helium, a workflow-aware serving framework that models agentic workloads as query plans and treats LLM invocations as first-class operators. Helium integrates proactive caching and cache-aware scheduling to maximize reuse across prompts, KV states, and workflows. Through these techniques, Helium bridges classic query optimization principles with LLM serving, achieving up to 1.56x speedup over state-of-the-art agent serving systems on various workloads. Our results demonstrate that end-to-end optimization across workflows is essential for scalable and efficient LLM-based agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。