解析大模型记忆机制，提升响应质量与效率
Cognitive Memory in Large Language Models
- 分感官、短时、长时三类记忆，分别对应输入、上下文和外部数据库
- 提出多种记忆压缩与管理方法，如低秩压缩、KV缓存共享、参数微调等
- 适合研究大模型推理优化与记忆增强的学者参考
本文探讨大型语言模型（LLMs）中的记忆机制，强调其在生成丰富上下文响应、减少幻觉及提升效率方面的重要性。将记忆分为感官记忆（对应输入提示）、短时记忆（处理即时上下文）和长时记忆（通过外部数据库或结构实现）。文本记忆涵盖获取（选择与摘要）、管理（更新、访问、存储与冲突解决）及利用（全文搜索、SQL查询、语义检索）。基于键值缓存（KV cache）的记忆方法包括选择策略（规律性摘要、评分法、特殊标记嵌入）与压缩技术（低秩压缩、KV合并、多模态压缩），以及卸载与共享注意力等管理策略。参数化记忆方法（如LoRA、TTT、MoE）将记忆转化为模型参数以提高效率；隐状态记忆方法（如分块机制、循环变压器、Mamba模型）通过结合RNN隐藏状态改进长文本处理。论文系统分析了大模型记忆机制，揭示其重要性并指明未来研究方向。
原文摘要 · Abstract (English)
This paper examines memory mechanisms in Large Language Models (LLMs), emphasizing their importance for context-rich responses, reduced hallucinations, and improved efficiency. It categorizes memory into sensory, short-term, and long-term, with sensory memory corresponding to input prompts, short-term memory processing immediate context, and long-term memory implemented via external databases or structures. The text-based memory section covers acquisition (selection and summarization), management (updating, accessing, storing, and resolving conflicts), and utilization (full-text search, SQL queries, semantic search). The KV cache-based memory section discusses selection methods (regularity-based summarization, score-based approaches, special token embeddings) and compression techniques (low-rank compression, KV merging, multimodal compression), along with management strategies like offloading and shared attention mechanisms. Parameter-based memory methods (LoRA, TTT, MoE) transform memories into model parameters to enhance efficiency, while hidden-state-based memory approaches (chunk mechanisms, recurrent transformers, Mamba model) improve long-text processing by combining RNN hidden states with current methods. Overall, the paper offers a comprehensive analysis of LLM memory mechanisms, highlighting their significance and future research directions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。