通过智能管理上下文，让手机端AI代理更省内存、更持久可用。
Efficient On-Device Agents via Adaptive Context Management
- 用特殊LoRA适配器压缩对话历史，生成结构化上下文对象。
- 上下文增长速度降低10到25倍，初始提示词体积减少6倍以上。
- 适合开发低延迟、高持久的本地AI助手，尤其适合资源受限设备。
手机端AI代理虽能提供个性化、低延迟服务，但受限于内存容量，可用上下文极短。这导致复杂交互与本地可行性之间存在矛盾。本文提出一种上下文高效的框架，包含三项协同优化：(1) 使用专用LoRA适配器动态管理记忆，将对话历史压缩为结构化的上下文状态对象；(2) 采用极简序列化格式表示工具模式，显著降低每个工具的令牌开销；(3) 仅在选择工具时才加载完整定义，实现按需加载。我们基于一个30亿参数的小型语言模型（SLM）实现该框架，并在复杂用户任务上与传统基线对比评估。结果表明，该代理在性能上达到或超过基线，同时上下文初始大小减少6倍以上，上下文增长速率降低10至25倍，具体取决于交互冗余程度，证明战略性上下文管理是实现强大且持续的本地AI的关键。
原文摘要 · Abstract (English)
On-device AI agents offer the potential for personalized, low-latency assistance, but their deployment is fundamentally constrained by limited memory capacity, which restricts usable context. This reduced practical context window creates a trade-off between supporting rich, stateful interactions with complex tool capabilities and maintaining on-device feasibility. We break this trade-off with a framework for context-efficient on-device agents, driven by three synergistic optimizations (1) a dynamic memory system using specialized LoRA adapters to distill conversational history into a compressed, and structured Context State Object; (2) a minimalist serialization format for tool schemas to minimize token overhead per tool; and (3) a just-in-time schema-passing mechanism that loads full tool definitions only upon tool selection. We instantiate this framework by adapting a 3B parameter SLM to context-efficient trajectories and rigorously evaluate it against a conventional baseline on complex user tasks. Our agent matches, or exceeds, the performance of a conventional baseline while dramatically compressing context, achieving more than a 6-fold reduction in initial system prompt context and a 10- to 25-fold reduction in context growth rate based on the interaction verbosity, demonstrating that strategic context management is key to unlocking capable and persistent on-device AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。