将长文本压缩为可移植的紧凑记忆，无需修改模型即可高效推理。
Latent Context Compilation: Distilling Long Context into Compact Portable Memory
- 用一次性LoRA模块编译上下文，生成可插拔的记忆令牌。
- 在16倍压缩比下仍保持细粒度推理能力，优于现有方法。
- 无需人工标注数据，适合部署在冻结模型上快速推理。
高效部署长上下文大模型面临两难：固定压缩难以泛化，而测试时训练需高昂合成数据成本且需修改模型权重，引入状态参数导致并发服务复杂。本文提出潜空间上下文编译（Latent Context Compilation），将上下文处理从适应转向编译。通过使用一次性LoRA模块作为编译器，将长上下文压缩为紧凑的缓冲令牌——无状态、可移植的记忆体，可直接接入冻结的基础模型。关键在于提出自对齐优化策略，无需依赖上下文相关的问答对。通过引入上下文无关的随机查询正则化上下文重建任务，强制压缩令牌位于模型原有指令遵循流形中。在Llama-3.1-8B上的实验表明，该方法在16倍压缩比下仍能保留精细细节与推理能力，有效解耦内存密度与模型参数，优于先前方法。
原文摘要 · Abstract (English)
Efficient long-context LLM deployment is stalled by a dichotomy between amortized compression, which struggles with out-of-distribution generalization, and Test-Time Training, which incurs prohibitive synthetic data costs and requires modifying model weights, creating stateful parameters that complicate concurrent serving. We propose Latent Context Compilation, a framework that fundamentally shifts context processing from adaptation to compilation. By utilizing a disposable LoRA module as a compiler, we distill long contexts into compact buffer tokens -- stateless, portable memory artifacts that are plug-and-play compatible with frozen base models. Crucially, we introduce a self-aligned optimization strategy that eliminates the need for synthetic context-relevant QA pairs. By regularizing context reconstruction task with context-agnostic random queries, we force compressed tokens to reside within the model's existing instruction-following manifold. Experiments with Llama-3.1-8B demonstrate that Latent Context Compilation preserves fine-grained details and reasoning capabilities where prior methods falter, effectively decoupling memory density from model parameters even at a 16x compression ratio.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。