arXiv:2605.06221cs.CL2026-05被引 1

通用长上下文预填充加速框架,适配各类模型且支持连续批处理。

UniPrefill: Universal Long-Context Prefill Acceleration via Block-wise Dynamic Sparsification

论文配图:UniPrefill: Universal Long-Context Prefill Acceleration via Block-wise Dynamic Sparsification
图 1 · 摘自论文原文
  • 基于块级动态稀疏化,在令牌层面直接加速模型计算。
  • 在并发请求增多时,首令牌时间最多提升2.1倍。
  • 兼容vLLM引擎,支持预填充与解码协同处理,适合部署优化。

随着大语言模型持续发展,其上下文长度需求日益增长。为提升长上下文推理效率,近年涌现出多种低复杂度混合架构,有效缓解了长上下文推理的计算负担。然而,现有长上下文预填充加速研究仍主要集中于稀疏注意力机制,仅在全注意力模型上达到最佳加速效果。当迁移至新型混合架构(如线性/全注意力混合或滑动窗口/全注意力混合)时,这些方法性能显著下降。此外,此类方法通常不兼容连续批处理,难以集成到vLLM等现代推理引擎中。为此,我们提出UniPrefill,一种适用于几乎所有模型架构的预填充加速框架,可在令牌级别直接加速模型计算。我们将UniPrefill实现为连续批处理算子,并扩展vLLM调度策略,原生支持预填充-解码协同处理及张量并行。UniPrefill在首令牌时间(TTFT)上最高实现2.1倍加速,且并发请求数越多,加速效果越明显。

原文摘要 · Abstract (English)

As large language models (LLMs) continue to advance rapidly, they are becoming increasingly capable while simultaneously demanding ever-longer context lengths. To improve the inference efficiency of long-context processing, several novel low-complexity hybrid architectures have recently been proposed, effectively alleviating the computational burden of long-context inference. However, existing research on long-context prefill acceleration remains predominantly focused on sparse attention mechanisms, which achieve their maximum speedup only on full-attention models. When transferred to emerging architectures--such as linear/full attention hybrids or sliding window/full attention hybrids--these prefill acceleration approaches suffer significant performance degradation. Furthermore, such methods are generally incompatible with continuous batching, making them difficult to integrate into modern inference engines such as vLLM. To this end, we propose UniPrefill, a prefill acceleration framework applicable to virtually any model architecture, which directly accelerates the model's computation at the token level. We further implement UniPrefill as a continuous batching operator and extend vLLM's scheduling strategy to natively support prefill-decode co-processing and tensor parallel for UniPrefill, enabling its seamless integration into vLLM. UniPrefill achieves up to 2.1x speedup in Time-To-First-Token (TTFT), with the acceleration becoming increasingly pronounced as the number of concurrent requests grows.

长上下文预填充加速vLLM连续批处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。