PReM动态保留与刷新上下文，提升长文本生成质量。
PReM: Learning What to Preserve and When to Refresh for Context Compression
- 用专用记忆层和特殊标记动态决定保留或刷新上下文。
- 在32K上下文下，16倍和32倍压缩时均优于基线模型。
- 适合需要高效长文本推理的场景，如复杂问答与摘要生成。
高效长上下文推理不仅关乎降低内存开销，更需在生成过程中持续保持有用上下文信息的可访问性。然而，现有压缩方法如键值缓存压缩和上下文压缩，往往过早决定保留哪些信息，或依赖外部压缩器，难以根据后续推理需求调整压缩内容。本文提出PReM（Preserve and Refresh Memory）框架，将长上下文作为模型内部逐层键值记忆，并学习何时保留、何时刷新。PReM使用专用记忆层进行选择决策，并引入特殊记忆标记<m>触发刷新。为训练该行为，提出分阶段刷新训练（Phase-Separated Refresh Training），使记忆选择与条件生成对齐，同时保持刷新间的连续性。在32K令牌上下文实验中，PReM在16倍和32倍压缩下均优于强基线，同时在答案质量与推理效率间保持良好平衡。
原文摘要 · Abstract (English)
Efficient long-context inference is not only about reducing memory cost, but also about keeping useful contextual evidence accessible as generation proceeds. However, existing compression-oriented approaches, such as key-value (KV) cache compression and context compression, often either make an early decision about which contextual information to keep or rely on an external compressor. Such designs make it difficult to adapt the compressed context to the evidence needed by later reasoning steps. This paper introduces PReM (Preserve and Refresh Memory), a context-compression framework that maintains the long context as the model's internal layer-wise KV memory and learns what to preserve and when to refresh it. Specifically, PReM uses a dedicated memory layer to make memory-selection decisions, and a special memory token <m> to trigger refreshes during generation. To train this behavior, PReM introduces Phase-Separated Refresh Training, aligning memory selection with memory-conditioned generation while preserving continuity across refreshes. Experiments with 32K-token contexts show that PReM outperforms strong baselines under both 16x and 32x compression, while maintaining a favorable balance between answer quality and inference efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。