用记忆检索提升问答效率，小模型也能逼近大模型性能。
Knowledge Access Beats Model Size: Memory Augmented Routing for Persistent AI Agents
- 用轻量模型+对话记忆检索，走低成本推理路径。
- 8B模型达30.5% F1，成本降96%，接近235B模型效果。
- 记忆比模型大小更重要，适合长期交互类应用。
生产级AI代理常面临用户重复性问题，高达47%的查询与历史交互语义相似，但每次处理仍消耗相同算力。我们提出一种内存增强推理框架，使用仅80亿参数的轻量模型，通过检索对话上下文实现低开销推理。无需额外训练或标注数据，该方法在152个LoCoMo和500个LongMemEval问题上取得30.5% F1，恢复了2350亿参数模型69%的性能，同时有效成本降低96%。值得注意的是，无记忆的2350亿参数模型仅得13.7% F1，甚至低于独立的80亿模型（15.4% F1），表明对用户特定查询而言，知识可访问性胜过模型规模。分析显示，路由机制本身可将96%查询导向小模型，但因自信幻觉导致准确率仅13.0% F1；而记忆通过引入用户特定信息提升正确性。随着记忆积累，重复话题覆盖上升，性能差距进一步缩小。结合混合检索（BM25 + 余弦相似度）可额外提升7.7 F1，证明检索质量直接影响系统表现。结果表明，对于持续型AI代理，记忆是决定准确率与效率的核心因素。
原文摘要 · Abstract (English)
Production AI agents frequently receive user-specific queries that are highly repetitive, with up to 47\% being semantically similar to prior interactions, yet each query is typically processed with the same computational cost. We argue that this redundancy can be exploited through conversational memory, transforming repetition from a cost burden into an efficiency advantage. We propose a memory-augmented inference framework in which a lightweight 8B-parameter model leverages retrieved conversational context to answer all queries via a low-cost inference path. Without any additional training or labeled data, this approach achieves 30.5\% F1, recovering 69\% of the performance of a full-context 235B model while reducing effective cost by 96\%. Notably, a 235B model without memory (13.7\% F1) underperforms even the standalone 8B model (15.4\% F1), indicating that for user-specific queries, access to relevant knowledge outweighs model scale. We further analyze the role of routing and confidence. At practical confidence thresholds, routing alone already directs 96\% of queries to the small model, but yields poor accuracy (13.0\% F1) due to confident hallucinations. Memory does not substantially alter routing decisions; instead, it improves correctness by grounding responses in retrieved user-specific information. As conversational memory accumulates over time, coverage of recurring topics increases, further narrowing the performance gap. We evaluate on 152 LoCoMo questions (Qwen3-8B/235B) and 500 LongMemEval questions. Incorporating hybrid retrieval (BM25 + cosine similarity) improves performance by an additional +7.7 F1, demonstrating that retrieval quality directly enhances end-to-end system performance. Overall, our results highlight that memory, rather than model size, is the primary driver of accuracy and efficiency in persistent AI agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。