LoRA可作为模块化知识记忆，实测验证其存储与组合能力。
Understanding LoRA as Knowledge Memory: An Empirical Analysis
- 用低秩适配（LoRA）构建可插拔的知识记忆模块
- 实证发现LoRA能有效存储和内部化新知识
- 适合需要低成本更新大模型知识的场景
预训练大语言模型的持续知识更新日益重要但挑战重重。尽管上下文学习（ICL）和检索增强生成（RAG）流行，却受限于上下文预算、成本及检索碎片化问题。本文突破依赖上下文的范式，探索使用低秩适配（LoRA）作为模块化知识记忆的参数化方法。虽有少数研究涉及此概念，但其存储容量与可组合性的基本机制仍不明晰。本文首次系统性地开展实证研究，覆盖从刻画存储容量、优化知识内化，到扩展多模块系统与长上下文推理评估的完整设计空间。不提出单一架构，而是提供洛拉记忆的实用操作边界。总体而言，我们的发现表明LoRA是继RAG和ICL后的互补型记忆轴，具有独特优势。
原文摘要 · Abstract (English)
Continuous knowledge updating for pre-trained large language models (LLMs) is increasingly necessary yet remains challenging. Although inference-time methods like In-Context Learning (ICL) and Retrieval-Augmented Generation (RAG) are popular, they face constraints in context budgets, costs, and retrieval fragmentation. Departing from these context-dependent paradigms, this work investigates a parametric approach using Low-Rank Adaptation (LoRA) as a modular knowledge memory. Although few recent works examine this concept, the fundamental mechanics governing its capacity and composability remain largely unexplored. We bridge this gap through the first systematic empirical study mapping the design space of LoRA-based memory, ranging from characterizing storage capacity and optimizing internalization to scaling multi-module systems and evaluating long-context reasoning. Rather than proposing a single architecture, we provide practical guidance on the operational boundaries of LoRA memory. Overall, our findings position LoRA as the complementary axis of memory alongside RAG and ICL, offering distinct advantages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。