让大模型外部记忆跨模型复用,关键在适配读取接口。
Cross-Model Memory Transfer via Target-Side Reader Adaptation

- 用轻量读取器适配目标模型,实现冻结记忆的跨模型迁移。
- 四分支双层读取器使跨模型问答准确率提升至38.8。
- 若读取接口兼容,无需训练即可直接使用,适配后效果更优。
提升大语言模型知识利用效率的方法通常分为两类:非参数检索灵活获取外部知识,但引入延迟和上下文开销,且与主干模型集成浅;参数化适配推理高效,但知识与权重耦合,难更新、难审计、难迁移。Engram式哈希记忆处于中间状态:将学习到的信息存于可寻址的外部表中,通过小型可学习读取器消费。本研究探讨当此类记忆从源模型迁移到目标模型时,是冻结的记忆本身更重要,还是目标侧读取器更关键。通过跨模型冻结记忆提取实验发现,学习到的记忆内容与正确寻址均重要,但只有当读取器与目标模型对齐时,记忆才真正可用。在下游问答任务中,双层四分支读取器几乎弥合了同模型与跨模型复用的差距,平均得分达38.8。此外,当提供方读取器与目标接口直接兼容时,冻结记忆无需目标侧训练即可发挥显著作用,而可选的读取器适配还能进一步提升性能。结果表明,只要目标模型具备兼容的读取接口,Engram可作为可复用的外部知识载体;若接口不匹配,目标侧适应能进一步优化对齐。
原文摘要 · Abstract (English)
Methods for improving knowledge use in large language models typically fall into two regimes. Non-parametric retrieval offers flexible access to external knowledge, but adds retrieval latency, context overhead, and only shallow integration with the backbone. Parametric adaptation is efficient at inference time, but entangles knowledge with model weights and can be hard to update, audit, or transfer. Engram-style hashed memory occupies a middle regime: it stores learned information in an external, addressable table, yet consumes that table through a small learned reader. This raises a basic question: when such a memory is moved across backbones, what matters more, the frozen memory itself or the target-side reader? We study this question through cross-model frozen-memory extraction, in which a memory trained on a source model is frozen and attached to a different target model, with only a lightweight reader trained. Ablations show that learned memory content and correct addressing both matter, but the transferred table becomes useful only through a reader aligned to the target model. In downstream question answering tasks, a dual-layer, four-branch reader nearly closes the gap between same-model and cross-model reuse, achieving an average score of 38.8 under our controlled evaluation protocol. Moreover, when the provider reader is directly compatible with the target interface, the frozen artifact can provide substantial utility without target-side training, while optional reader adaptation yields further improvement. These results suggest that Engram can serve as a reusable external knowledge artifact, provided that the target has access to a compatible reader interface; target-side adaptation can further improve alignment when direct reader reuse is insufficient.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。