用记忆复用降低大模型推理成本，智能分流难问题。
MemBoost: A Memory-Boosted Framework for Cost-Aware LLM Inference
- 构建可成长记忆库，复用历史回答减少重复计算
- 仅对不确定请求调用强模型，节省70%以上高成本调用
- 适合需要低成本、高响应的交互式大模型服务
大型语言模型（LLMs）虽性能强大，但在实际服务中推理成本高昂，尤其在用户与会话间存在重复或近似查询时更为显著。本文提出MemBoost框架，通过轻量级模型复用先前生成的答案并检索相关支持信息，实现低成本推理；同时对复杂或不确定的请求智能升级至更强模型处理。不同于传统检索增强生成仅针对单次响应，MemBoost专为交互场景设计，支持答案复用、持续记忆增长与成本感知路由。在多种模型和模拟工作负载下的实验表明，该方法显著减少昂贵的大模型调用次数，整体推理成本大幅下降，同时保持与强模型基线相当的回答质量。
原文摘要 · Abstract (English)
Large Language Models (LLMs) deliver strong performance but incur high inference cost in real-world services, especially under workloads with repeated or near-duplicate queries across users and sessions. In this work, we propose MemBoost, a memory-boosted LLM serving framework that enables a lightweight model to reuse previously generated answers and retrieve relevant supporting information for cheap inference, while selectively escalating difficult or uncertain queries to a stronger model. Unlike standard retrieval-augmented generation, which primarily grounds a single response, MemBoost is designed for interactive settings by supporting answer reuse, continual memory growth, and cost-aware routing. Experiments across multiple models under simulated workloads show that MemBoost substantially reduces expensive large-model invocations and overall inference cost, while maintaining high answer quality comparable to the strong model baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。