用信息瓶颈原理动态优化大模型记忆，提升长期任务表现
MemFly: On-the-Fly Memory Optimization via Information Bottleneck
- 基于信息瓶颈思想，无梯度优化实现记忆压缩与保留
- 多路径检索机制支持复杂多跳查询，准确率显著提升
- 适合需要长期记忆的智能体系统，如对话机器人、规划助手
长期记忆使大型语言模型代理能够通过历史交互完成复杂任务。然而，现有框架在高效压缩冗余信息与保持下游任务精确检索之间面临根本矛盾。为此，我们提出 MemFly，一个基于信息瓶颈原理的框架，实现大模型记忆的实时演化。该方法通过无梯度优化器最小化压缩熵、最大化相关性熵，构建分层记忆结构以实现高效存储。为充分释放 MemFly 潜力，我们设计了一种混合检索机制，无缝融合语义、符号和拓扑路径，并引入迭代优化以应对复杂多跳查询。全面实验表明，MemFly 在记忆连贯性、回复保真度和准确性方面显著优于当前最优基线。
原文摘要 · Abstract (English)
Long-term memory enables large language model agents to tackle complex tasks through historical interactions. However, existing frameworks encounter a fundamental dilemma between compressing redundant information efficiently and maintaining precise retrieval for downstream tasks. To bridge this gap, we propose MemFly, a framework grounded in information bottleneck principles that facilitates on-the-fly memory evolution for LLMs. Our approach minimizes compression entropy while maximizing relevance entropy via a gradient-free optimizer, constructing a stratified memory structure for efficient storage. To fully leverage MemFly, we develop a hybrid retrieval mechanism that seamlessly integrates semantic, symbolic, and topological pathways, incorporating iterative refinement to handle complex multi-hop queries. Comprehensive experiments demonstrate that MemFly substantially outperforms state-of-the-art baselines in memory coherence, response fidelity, and accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。