通过混合离线在线缓存,显著提升长文本推理速度。
RedKnot-MLA: Multi-Head Offline-Online Reuse for DeepSeek-V4 Long-Context Serving

- 用单一隐式键值流实现多头注意力,节省内存
- 离线预处理+在线重定位,256K上下文下速度提升2-3.8倍
- 适合需要高效长文本处理的AI服务系统
多头隐式注意力(MLA)通过单一打包的隐式键值流暴露多个逻辑查询头,具有内存效率优势,但打破了传统按头缓存的物理边界。本文提出RedKnot-MLA系统,基于DeepSeek-V4实现头感知复用。每个不可变文档在标准位置零处离线处理,保留局部头贡献为MLA-Off;服务时,查询侧RoPE重定位恢复文档请求位置,小规模全局头集与受保护局部标记行重新计算为MLA-Online,两路径合并后经单一共享输出投影。打包的MLA隐式表示始终不拆分。DeepSeek-V4-Flash使用37层、56/8局部/全局头划分,理论逻辑头行上限达75.29%;Pro-0813配置使用55层、112/16头,达78.89%。冻结模型显示首次令牌时间(TTFT)提速2.02-3.84倍。在256K上下文下,三数据集综合研究显示F1提升3.24个百分点,EM提升4.16点,主要算子算术量减少78.7-79.5%,单个数据集下降2.81 F1点。另报告256K热工件QPS约2.0倍,因无原始并发追踪,列为初步证据。
原文摘要 · Abstract (English)
Multi-head latent attention (MLA) exposes many logical query heads through one packed latent KV stream. This representation is memory efficient, but it removes the physical per-head cache boundary assumed by conventional head-wise reuse. We present our system, a DeepSeek-V4 realization of RedKnot's head-aware reuse principle. Each immutable document is processed offline at canonical position zero; certified Local-head contributions are retained as MLA-Off. At serving time, query-side RoPE relocation restores the document's request position, a small Global-head set and protected Local token rows are recomputed as MLA-Online, and the two paths are merged before a single shared output projection. The packed MLA latent is never split. DeepSeek-V4-Flash uses 37 reusable layers and a 56/8 Local/Global partition, giving a 75.29% analytic logical head-row ceiling; the Pro-0813 profile uses 55 layers and 112/16 heads, giving 78.89%. Frozen Flash operating points show hot-artifact TTFT speedups of 2.02-3.84x. At 256K, the archived three-dataset study reports an aggregate F1 change of +3.24 percentage points, an EM change of +4.16 points, and a 78.7-79.5% analytic major-operator arithmetic saving, while one dataset decreases by 2.81 F1 points. A separate author-reported 256K hot-artifact QPS measurement is approximately 2.0x; because its raw concurrency trace is not included in this bundle, we mark it as preliminary rather than archived evidence. We describe the factorization, position repair, token-row closure, sparse-MoE support, TP8 integration, and the measurement boundaries needed to interpret these results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。