将上下文压缩为可管理的模块化记忆，提升模型推理效率与鲁棒性。
Context Distillation as Latent Memory Management
- 将每个上下文转化为独立LoRA适配器，构建可显式选择的记忆库
- 通过查询检索匹配记忆，自适应激活最相关适配器，性能显著优于基线
- 引入自门控机制和缓存共享，减少冗余计算，适合多任务推理场景
上下文蒸馏将上下文信息压缩到模型参数中，但现有方法常忽略多个蒸馏后的潜在记忆如何在非理想设置下存储、检索和安全激活。本文将上下文蒸馏视为潜在记忆管理问题：将每个上下文蒸馏为独立的LoRA适配器，形成模块化记忆库，实现显式记忆选择。给定查询时，框架检索候选记忆，路由查询至最适配的适配器，并使用自门控机制决定是否激活潜在记忆。为提高效率，进一步引入缓存共享以降低推理阶段的管理开销。实验表明，该方法在具有检索能力的场景下显著优于基线；自门控机制通过禁用不必要的潜在记忆提升了鲁棒性。
原文摘要 · Abstract (English)
Context distillation compresses contextual information into model parameters, yet existing methods often ignore how multiple distilled latent memories should be stored, retrieved, and safely activated in non-oracle settings. We formulate context distillation as a latent memory management problem. We distill each context into an independent LoRA adapter, forming a modular memory bank that enables explicit memory selection. Given a query, our framework retrieves candidate memories, routes the query to the most suitable adapter, and uses a Self-Gating mechanism to decide whether latent memory should be activated. To improve efficiency, we further introduce cache sharing to reduce management overhead during inference. Experiments show that our method substantially outperforms baselines with retrieval, while Self-Gating improves robustness by deactivate unnecessary latent memories.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。