让手机端AI代理更快，速度提升61%且不丢精度。
Agent-X: Full Pipeline Acceleration of On-device AI Agents
- 重写提示词，利用前缀缓存加速推理
- 无LLM的推测性生成，实现快速出词
- 适合部署在手机等边缘设备的AI代理
基于大模型的智能体在各类任务中表现卓越，但在边缘设备上存在高延迟问题。我们提出Agent-X，一个纯软件、不损失精度的加速框架，可同时加速设备端智能体工作流中的预填充和解码阶段。其两大核心组件：一是针对智能体输入特征重写提示词，以更好利用前缀缓存；二是实现无需大模型的推测性解码，大幅加快生成速度且开销极小。在典型智能体任务中，Agent-X 在真实系统中实现1.61倍的端到端加速，且无准确率下降，可无缝集成至现有设备端AI代理系统。据我们所知，这是首个系统性分析并消除设备端智能体延迟瓶颈的工作。
原文摘要 · Abstract (English)
LLM-based agents deliver state-of-the-art performance across tasks but incur high end-to-end latency on edge devices. We introduce Agent-X, a software-only, accuracy-preserving framework that accelerates both the prefill and decode stages of on-device agent workloads. Agent-X's two key components rewrite prompts to leverage prefix caching tailored to agent-specific input-token patterns and enable LLM-free speculative decoding for fast token generation with minimal overhead. On representative agentic workloads, Agent-X achieves a 1.61x end-to-end speedup in real systems with no accuracy loss and can be seamlessly integrated into existing on-device AI agents. To the best of our knowledge, ours is the first to systematically characterize and eliminate latency bottlenecks in on-device agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。