不训练模型,用固态硬盘存知识库,让大模型推理更快更准。
TF-Engram: A Train-Free Engram with SSD-Backed Memory for Large Language Models

- 离线构建特定短语记忆,用固态硬盘存海量数据
- 在0.6B小模型上得分从57.6提升到59.4,优于参数匹配的LoRA
- 无需训练、不占显存,适合部署在资源受限场景
大型语言模型将事实知识和领域模式隐式存储于密集的Transformer参数中,扩展知识需通过预训练、微调、检索增强或更长上下文,成本高昂。基于记忆的Engram方法提供紧凑的隐藏状态注入路径,但现有依赖GPU内存的设计常采用哈希压缩,导致无关短语冲突,降低语义保真度。本文提出TF-Engram,一种无需训练的Engram系统:从外部语料离线构建短语级语义记忆,跨GPU-DRAM-SSD层级存储大规模记忆表,并使用早退引导的预测预取技术隐藏外部内存延迟。在Qwen3-0.6B上,TF-Engram将平均下游得分从57.6提升至59.4,优于冻结主干和参数匹配的LoRA基线。系统评估显示,大容量TF-Engram表可实现适度离线开销,基于SSD的存储显著降低显存需求,预测预取恢复了大部分外部内存访问带来的吞吐损失。结果表明,静态短语记忆可作为可扩展、免训练、低开销的组件集成至大模型推理中。
原文摘要 · Abstract (English)
Large Language Models (LLMs) store factual knowledge and domain-specific patterns implicitly in dense Transformer parameters, making knowledge expansion costly through pretraining, fine-tuning, retrieval augmentation, or longer contexts. Engram-style memory offers a compact hidden-state injection pathway, but existing GPU-resident designs often rely on hash-based compression, causing unrelated phrases to collide in shared slots and weakening phrase-level semantic fidelity. We present TF-Engram, a train-free Engram system that constructs phrase-specific semantic memory offline from external corpora, stores large memory tables across a GPU--DRAM--SSD hierarchy, and uses Early-Exit Guided Predictive Prefetching to hide external-memory latency during autoregressive decoding. On Qwen3-0.6B, TF-Engram improves the average downstream score from 57.6 to 59.4, outperforming both the frozen backbone and a parameter-matched LoRA baseline. System evaluation shows that large TF-Engram tables can be built with moderate offline cost, SSD-backed storage substantially reduces GPU memory demand, and predictive prefetching recovers much of the throughput loss caused by external memory access. These results demonstrate that static phrase memory can be integrated into LLM inference as a scalable, train-free, and low-overhead system component.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。