通过前瞻稀疏注意力,让大模型处理超长文本时内存占用降90%。
FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention
- 用神经记忆索引提前预测需要的上下文,只保留关键片段。
- 在100万上下文下,显存减少90%,推理速度提升2.8倍。
- 适合高并发超长文本服务场景,如智能客服、法律分析。
传统大模型在解码时需完整加载所有键值缓存(KV cache),导致超长上下文服务严重受限于GPU显存。本文提出一种新型推理范式——前瞻稀疏注意力(Lookahead Sparse Attention, LSA),基于DeepSeek-V4架构构建神经记忆索引器。不同于被动访问全部历史标记,LSA主动预测未来上下文需求,仅将查询相关的关键KV块保留在GPU内存中。关键创新在于采用无主干模型的解耦训练策略:将索引器建模为标准双编码器结构,使用标准检索训练框架独立训练,无需加载庞大的主干模型进入显存。实验表明,该‘少即是多’范式显著提升服务效率,并有效降低长时依赖任务中的注意力噪声。在主流长上下文评测集(LongBench-v2、LongMemEval、RULER)上,FM-DS-V4将平均物理KV缓存大小压缩至全上下文基线的13.5%,同时保持或轻微提升下游准确率(平均+0.6%绝对提升)。在100万上下文长度下,每解码一个词的计算量降至基线的0.30倍,GPU KV缓存从3.73 GB降至0.37 GB,实现8×H20 GPU部署下的聚合吞吐量提升2.8倍、并发能力提升2.7倍。
原文摘要 · Abstract (English)
Conventional LLMs keep the full KV cache loaded during decoding, causing a severe GPU memory bottleneck for ultra-long context serving. In this report, we propose \textbf{Lookahead Sparse Attention (LSA)}, a novel inference paradigm powered by a Neural Memory Indexer built upon the DeepSeek-V4 architecture. Rather than passively attending to all historical tokens, LSA proactively predicts future context demands and preserves only the query-critical KV chunks in the GPU memory. Crucially, we instantiate this architecture via a \textbf{backbone-free decoupled training} strategy. By formulating the indexer as a standard dual-encoder architecture, we train it independently using standard retrieval training frameworks without ever loading the massive backbone model into GPU memory. We demonstrate that this ``less is more'' paradigm significantly maximizes serving efficiency while acting as an effective attention denoiser in tasks that rely on long-term global memory. Across primary long-context evaluation suites (e.g., LongBench-v2, LongMemEval, and RULER), \texttt{FM-DS-V4} compresses the average physical KV cache footprint down to merely 13.5\% of the full-context baseline, while consistently preserving or slightly elevating downstream accuracy (+0.6\% absolute margin on average). At 1M context, per-decode-token compute drops to 0.30$\times$ of the baseline and GPU KV cache shrinks by 90\% (3.73$\to$0.37 GB), translating into \textbf{2.8$\times$ aggregate throughput and 2.7$\times$ concurrency gains} in PD-disaggregated serving on 8$\times$H20 GPUs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。