arXiv:2605.20724cs.IR2026-05

让对话AI突破上下文限制,实现无边界记忆。

CALMem : Application-Layer Dual Memory for Conversational AI

  • 在应用层构建双记忆系统:滑动窗口历史与可写结构化事实
  • 动态注入过往对话,上下文压力越大注入越深
  • 支持会话内检索已压缩内容,适合长时对话场景

大语言模型受限于固定上下文窗口,导致对话连续性中断。现有方案或需修改模型、或依赖特定平台,无法解决压缩后历史不可恢复的问题。我们提出CALMem(对话应用层双重记忆),无需修改底层模型即可实现近乎无限的有效上下文。其包含两个互补子系统:基于滑动窗口向量嵌入的事件记忆层,以及代理可写结构化事实的语义记忆层。通过自适应注入机制MOIM,根据上下文压力动态检索并注入相关历史,注入深度与压力成反比。关键创新在于会话内检索:即使当前会话中部分对话被压缩,仍可搜索。系统以纯应用层形式在生产级Rust代码中实现,对提供商无绑定,关闭时零开销退化为原生LLM行为。详细阐述架构设计、性能特征及决策权衡。

原文摘要 · Abstract (English)

Large language models (LLMs) operate within fixed context windows that fundamentally limit conversational continuity. When context fills, compaction discards history irreversibly; when sessions end, all memory resets to zero. Existing solutions-larger context windows, retrieval-augmented generation for knowledge bases, and memory-augmented architectures such as MemGPT-either require model modification, impose provider lock-in, or do not address the compaction continuity problem. We present CALMem (Conversational Application-Layer Memory), an application-layer dual memory architecture that gives LLM-based conversational assistants virtually unbounded effective context without any modification to the underlying model. CALMem combines two complementary memory subsystems: an episodic memory layer built on sliding-window vector embeddings of conversation history, and a semantic memory layer of agent-writable structured facts. A token-budget-adaptive injection mechanism, called the MOIM (Message of Injected Memory), automatically retrieves and injects relevant past context each turn, scaling injection depth inversely with context pressure. A key contribution is intra-session retrieval: compacted away turns from the current session remain searchable, closing a gap unaddressed by prior work. The system is implemented as a pure application layer in a production Rust codebase, is provider-agnostic, and degrades to original LLM behaviour with zero overhead when disabled. We describe the architecture, design decisions, and performance characteristics, and analyse the trade-offs that guided each implementation choice.

对话系统记忆机制上下文扩展

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。