跨模型复用注意力缓存,加速大模型切换时的预填充过程。
Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse

- 发现不同模型间键值对存在线性关系,可构建闭式映射器复用缓存。
- 在6组模型对中,98%准确率保留,提速2.7至25倍。
- 适合需要频繁切换模型的生产部署场景,如成本-质量动态调度。
生产环境中常在同家族不同尺寸模型间切换以实现成本-质量平衡、对话中切换或流量路由,但每次切换均需重新进行预填充。本文提出跨模型KV缓存迁移,使接收端可复用源模型的KV缓存,跳过预填充。研究发现,在头数与每头维度匹配的模型对中,跨模型KV具有显著线性结构:以Qwen3 14B→32B为例,单个源层可解释目标键向量56%的方差和值向量32%的方差,多层叠加后分别提升至79%和65%。基于此,设计一种闭式岭回归映射器,按头独立操作,分三步:首先选取对目标层预测力最强的前k个源层并拼接其KV;其次在映射前剥离RoPE,实现位置无关,适配不同上下文长度;最后在500条1024标记的FineWeb-Edu样本上拟合。结果显示,六组模型对中,四组保持73%-98%的独立预填充准确率,两组明显下降;使用非线性MLP可恢复最高+37个百分点的HellaSwag保留率。该映射器运行速度比重算预填充快2.7至25倍,且在多轮切换中稳定,具备实际可行性。
原文摘要 · Abstract (English)
Production deployments often swap between different-sized models in a family for cost-quality cascading, mid-conversation switching, and routing, and each swap forces the receiver to repay the prefill from scratch. We propose cross-model KV cache transfer, where the receiver reuses the source's KV cache, skipping prefill. We find that cross-model KV has substantial linear structure across matched-KV pairs, where source and target share KV head count and per-head dimension. On Qwen3 14B->32B, one source layer explains 56% of variance in the target's keys and 32% in values, rising to 79% and 65% with multiple source layers. Building on this, we design a closed-form ridge mapper that operates per head and proceeds in three steps. First, for each target layer we select the top-k most predictive source layers and concatenate their KV as input. Second, we strip RoPE from the keys before mapping, so the fit is position-free and reusable across context lengths. Third, we fit ridge regression on a small calibration set of 500 FineWeb-Edu sequences of 1,024 tokens each. Surprisingly, across six pairs in three families, this linear mapper retains 73-98% of the receiver's standalone-prefill accuracy on four pairs, while two degrade sharply. A nonlinear MLP recovers up to +37 pp HellaSwag retention on the failures. The mapper runs 2.7-25x faster than re-prefill and remains stable across multi-turn handoff, making cross-model KV cache transfer practical.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。