给小模型加记忆系统，让大模型推理更快更准。
Speculate with Memory: Lossless Acceleration for LLM Agents
- 用三个记忆模块记录历史轨迹，提升预测能力。
- 动作预测准确率提升19%~39%，观测预测提速2.5倍。
- 无需额外时间，适合各类低成本推理场景。
推测执行通过使用更小、更便宜的模型，在环境空闲时预判并提前启动下一步来加速大语言模型智能体。然而，现有推测器无状态，任务间信息全丢，无法从经验中学习。本文为推测器引入三种在线记忆系统：对比转移表记录动作序列统计，情景记忆检索上下文相似片段，混淆追踪器抑制重复错误。在六个基准测试上评估了三种推测类型：动作预测、观测预测和链式预测。带记忆的推测在动作预测上相对准确率提升19%–39%，在具有重复动作空间的观测预测任务上最高提升达2.5倍。性能随记忆积累持续增长，并在不同成本的推测模型间具备泛化性。所有推测均在空闲时段运行，零额外耗时，且执行轨迹与非推测模式完全一致。
原文摘要 · Abstract (English)
Speculative execution accelerates LLM agents by using a smaller, cheaper model to predict and pre-launch the next step while the environment is idle. However, existing speculators are stateless and discard all information between tasks, preventing prediction quality from improving with experience. We equip the speculator with three online memory systems that learn from past agent trajectories: a contrastive transition table tracking action-sequence statistics, an episodic memory retrieving contextually similar segments, and a confusion tracker suppressing recurring errors. We evaluate this approach on six benchmarks spanning three speculation types: action prediction, observation prediction, and chained prediction. Memory-augmented speculation yields a 19--39\% relative accuracy improvement on action prediction and up to a $2.5\times$ increase on observation prediction tasks with repetitive action spaces. These gains grow continuously as memory accumulates and generalize across speculator models of varying cost. All speculation is lossless because it runs during idle time at zero added wall-clock cost, and the actor's trajectory is identical to non-speculative execution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。