让大模型知识可读可改,通过外部记忆库提升透明度和更新效率
ExplicitLM: Decoupling Knowledge from Parameters via Explicit Memory Banks
- 用百万级外部记忆库存事实,支持直接查看修改
- 在低数据场景下性能提升3.62倍,知识任务准确率最高提升43.67%
- 结合认知理论分层管理知识,实现可解释的动态更新
大语言模型因知识隐式存储于参数中,存在知识滞后与不可解释问题,难以精准更新与透明推理。我们提出ExplicitLM,采用百万级外部记忆库以可读的词元序列存储知识,支持直接检查与修改。设计可微双阶段检索机制:先通过产品键分解实现粗粒度过滤(复杂度从$\ ext{O}(N \cdot |I|)$降至$\ ext{O}(\sqrt{N} \cdot |I|)$),再用Gumbel-Softmax进行细粒度匹配,支持端到端训练。受双系统认知理论启发,将知识分为20%冻结的显式事实与80%可学习的隐式模式,通过指数移动平均维持稳定。实验显示,ExplicitLM在知识密集型任务上相较标准Transformer最高提升43.67%,低数据场景(1万样本)下性能提升3.62倍。分析表明,记忆检索与性能强相关,正确预测的命中率高出49%。相比检索模块冻结的RAG系统,本架构联合优化,实现可解释、可更新与高性能兼备。
原文摘要 · Abstract (English)
Large language models suffer from knowledge staleness and lack of interpretability due to implicit knowledge storage across entangled network parameters, preventing targeted updates and reasoning transparency. We propose ExplicitLM, a novel architecture featuring a million-scale external memory bank storing human-readable knowledge as token sequences, enabling direct inspection and modification. We design a differentiable two-stage retrieval mechanism with efficient coarse-grained filtering via product key decomposition (reducing complexity from $\mathcal{O}(N \cdot |I|)$ to $\mathcal{O}(\sqrt{N} \cdot |I|)$) and fine-grained Gumbel-Softmax matching for end-to-end training. Inspired by dual-system cognitive theory, we partition knowledge into frozen explicit facts (20%) and learnable implicit patterns (80%), maintained through Exponential Moving Average updates for stability. ExplicitLM achieves up to 43.67% improvement on knowledge-intensive tasks versus standard Transformers, with 3.62$\times$ gains in low-data regimes (10k samples). Analysis shows strong correlations between memory retrieval and performance, with correct predictions achieving 49% higher hit rates. Unlike RAG systems with frozen retrieval, our jointly optimized architecture demonstrates that interpretable, updatable models can maintain competitive performance while providing unprecedented knowledge transparency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。