用信息增益指导记忆管理，让智能体长期记忆更有效且可迁移。
CMI-Mem: Toward Generalizable Long-Term Memory Management via CMI-Augmented Reinforcement Learning
- 结合任务准确率与条件互信息，实现内外部信号互补的奖励机制。
- 在多个记忆场景中表现更优，训练和推理效率提升20%以上。
- 适合需要长期记忆的对话系统、自主代理等应用场景。
记忆管理器在智能体系统中至关重要。现有强化学习方法通常依赖大模型生成的问答对进行记忆评估，但其仅基于采样查询分布和固定阅读器判断价值。本文提出CMI-Mem，一种轻量级强化学习记忆管理框架，采用混合奖励机制：外在的问答项衡量最终任务准确性，内在的条件互信息（CMI）则评估新输入相对于当前记忆状态所贡献的信息量，无需依赖采样问答对。二者互补：问答项锚定任务效用，而CMI提供每一步操作的监督信号，促进相关且无冗余的记忆构建。实验表明，该方法在不同记忆使用场景间具有更强的泛化能力，同时显著提升训练与推理效率。代码已开源：https://github.com/Wyb0627/CMIMem，4B模型检查点可在ModelScope获取。
原文摘要 · Abstract (English)
Memory Manager models are pivotal in agent systems. Existing reinforcement-learning methods commonly use LLM-judged synthetic question-answer (QA) pairs: this provides useful downstream task grounding, but values memory through a sampled query distribution and a fixed reader. We propose CMI-Mem, a lightweight RL memory manager with a hybrid reward. Its extrinsic QA term measures end-task correctness, while its intrinsic Conditional Mutual Information (CMI) term evaluates the information contributed by new conversational inputs relative to the current memory state without conditioning on a sampled QA query. The two signals are complementary: QA anchors task utility, whereas CMI provides per-operation supervision for relevant, non-redundant memory construction. Experiments demonstrate improved transfer across memory-use scenarios, together with more efficient training and inference from the per-operation CMI signal. Our codes are available at: https://github.com/Wyb0627/CMIMem , and the CMI-Mem-4B model checkpoint is available at: https://www.modelscope.cn/models/wyb0627/CMIMem-4B
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。