arXiv:2604.20819cs.LGcs.DC2026-04

提出流式调度方法,让单卡运行百亿词长文本注意力计算不崩溃。

Stream-CQSA: Avoiding Out-of-Memory in Attention Computation via Flexible Workload Scheduling

论文配图:Stream-CQSA: Avoiding Out-of-Memory in Attention Computation via Flexible Workload Scheduling
图 1 · 摘自论文原文
  • 基于循环共识集理论分解注意力,实现无误差分块计算
  • 可在任意内存预算下执行,支持单卡处理百亿词序列
  • 无需近似、无需跨设备通信,适合长序列建模场景

长上下文大模型的可扩展性受限于自注意力机制的二次内存开销,常导致现代硬件出现内存溢出(OOM)失败。现有方法虽将内存复杂度降至近线性,但假设查询、键、值张量能完整放入设备内存。本文通过引入源自循环共识集(CQS)理论的CQS Divide操作,将注意力分解为一组独立子序列计算,其重组结果与全序列注意力完全一致。基于此,提出Stream-CQSA内存自适应调度框架,将注意力划分为符合任意内存预算的子问题。该设计将注意力从逻辑上单一的操作转变为可调度任务集合,实现跨设备灵活执行且无需设备间通信。实验表明,内存消耗可预测地随规模增长,且在不改变注意力数学定义、不引入近似误差的前提下,单个GPU即可通过流式处理执行百亿词序列的精确注意力。

原文摘要 · Abstract (English)

The scalability of long-context large language models is fundamentally limited by the quadratic memory cost of exact self-attention, which often leads to out-of-memory (OOM) failures on modern hardware. Existing methods improve memory efficiency to near-linear complexity, while assuming that the full query, key, and value tensors fit in device memory. In this work, we remove this assumption by introducing CQS Divide, an operation derived from cyclic quorum sets (CQS) theory that decomposes attention into a set of independent subsequence computations whose recomposition yields exactly the same result as full-sequence attention. Exploiting this decomposition, we introduce Stream-CQSA, a memory-adaptive scheduling framework that partitions attention into subproblems that fit within arbitrary memory budgets. This recasts attention from a logically monolithic operation into a collection of schedulable tasks, enabling flexible execution across devices without inter-device communication. Experiments demonstrate predictable memory scaling and show that exact attention over billion-token sequences can be executed on a single GPU via streaming, without changing the underlying mathematical definition of attention or introducing approximation error.

注意力机制内存优化长序列建模流式计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。