让大模型缓存文档时忽略位置，提升长文本检索效率
SemPIC: Learning Semantic Position-Independent KV Caches
- 用LoRA微调生成器，让文档嵌入适应不同上下文
- 在4个任务上平均准确率从0.53提升至0.60
- 适合需要重复读取文档的智能体和长文本场景
长上下文检索与智能体任务中，相同文档在不同指令、历史和顺序下被反复使用。前缀缓存无法利用这种重复性，而传统的位置无关缓存（PIC）因缺乏未来上下文信息仍不可靠。诊断发现，学习边界条件可显著降低重用块边界的注意力偏差，但内部和任务级残差仍存在，促使我们改进文档表示本身。本文提出SemPIC，通过行为蒸馏训练一个带LoRA的写入器，在各层生成原生文档键值对（KV），同时保持预训练解码器不变作为读取器。适配过程仅限离线缓存构建，保留标准缓存接口和命中解码路径。此外引入KV梯度检查点技术，减少峰值训练内存且不中断缓存键值的梯度流动。在三个模型和四个任务上，SemPIC将微F1平均从0.53提升至0.60，接近全重计算的0.62。代码见：https://github.com/jn12-29/SemPIC
原文摘要 · Abstract (English)
Long-context retrieval and agentic workloads repeatedly reuse the same documents under changing instructions, histories, and document orders. Prefix caching cannot exploit this reuse, while position-independent caching (PIC) remains unreliable because independently compiled KV states lack the future context in which they will be consumed. Our diagnostics show that a learned boundary-conditioned baseline sharply reduces attention deviation near reusable-block boundaries but leaves interior and task-level residuals, motivating adaptation of the document representation itself. We present \emph{SemPIC}, which trains a LoRA-enabled Writer to compile native per-layer document KVs through behavioral distillation while retaining the pretrained decoder as an unchanged Reader. Adaptation is confined to offline cache construction, preserving the standard KV interface and cache-hit decoding path. We further introduce KV Gradient Checkpointing, which reduces peak training memory without severing gradients through cached KVs. Across three models and four tasks, SemPIC raises mean micro-F1 over KV Packet from 0.53 to 0.60, approaching Full Recompute at 0.62. Code: https://github.com/jn12-29/SemPIC
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。