Pie让大模型应用自定义推理流程，提升复杂任务效率。
Pie: A Programmable Serving System for Emerging LLM Applications
- 将生成流程拆解为可编程的服务模块，由用户代码控制
- 在代理工作流上实现1.3至3.4倍的吞吐提升，延迟更低
- 适合需要自定义推理逻辑的复杂大模型应用开发者
新兴的大语言模型应用涉及多样的推理策略和智能体工作流，对传统基于单体令牌生成循环的部署系统构成挑战。本文提出Pie，一种面向灵活性与高效性的可编程大模型服务系统。Pie将传统生成循环分解为细粒度的服务处理器，通过API暴露，并将生成过程控制权交由用户编写的程序（称为inferlets）实现。这使得应用可在不修改服务系统的情况下，自主实现新的键值缓存策略、定制化生成逻辑，并无缝集成计算与I/O操作。Pie采用WebAssembly执行inferlets，获得轻量级沙箱优势。评估表明，Pie在标准任务上达到当前最优性能（3-12%延迟开销），在智能体工作流上显著提升延迟与吞吐（1.3x-3.4x更高），支持应用级优化。
原文摘要 · Abstract (English)
Emerging large language model (LLM) applications involve diverse reasoning strategies and agentic workflows, straining the capabilities of existing serving systems built on a monolithic token generation loop. This paper introduces Pie, a programmable LLM serving system designed for flexibility and efficiency. Pie decomposes the traditional generation loop into fine-grained service handlers exposed via an API and delegates control of the generation process to user-provided programs, called inferlets. This enables applications to implement new KV cache strategies, bespoke generation logic, and seamlessly integrate computation and I/O-entirely within the application, without requiring modifications to the serving system. Pie executes inferlets using WebAssembly, benefiting from its lightweight sandboxing. Our evaluation shows Pie matches state-of-the-art performance on standard tasks (3-12% latency overhead) while significantly improving latency and throughput (1.3x-3.4x higher) on agentic workflows by enabling application-specific optimizations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。