arXiv:2512.06457cs.LGeess.SP2025-12被引 4

通过融合与提前终止,实现高效低功耗的Transformer注意力加速。

BitStopper: An Efficient Transformer Attention Accelerator via Stage-fusion and Early Termination

  • 将预测阶段融入执行阶段,按位串行复用减少内存访问。
  • 在不依赖预测器前提下,实现2.03倍速度提升和2.4倍能效优化。
  • 适合追求低延迟、高能效的推理部署场景,尤其适用于边缘设备。

基于注意力的大语言模型推动了现代AI应用的发展,但自注意力的二次方计算开销带来了巨大的算力与内存负担。动态稀疏(DS)注意力虽能缓解此问题,但其硬件效率受限于额外的预测阶段及高额内存流量。本文提出BitStopper,一种无需稀疏预测器的细粒度软硬件协同设计。首先,引入比特串行使能阶段融合(BESF)机制,通过逐步终止无意义标记并融合预测阶段至执行阶段,减少内存访问。其次,设计轻量级自适应标记选择(LATS)策略,配合比特级稀疏推测。第三,采用比特级异步处理(BAP)策略,在按需获取比特粒度内存时提升计算利用率。最终,构建了精细架构,将理论复杂度降低转化为实际性能提升。大量评估表明,相比现有最先进(SOTA)Transformer加速器,BitStopper相较Sanger和SOFA分别实现2.03倍和1.89倍的速度提升,能效分别提高2.4倍和2.1倍。

原文摘要 · Abstract (English)

Attention-based large language models (LLMs) have transformed modern AI applications, but the quadratic cost of self-attention imposes significant compute and memory overhead. Dynamic sparsity (DS) attention mitigates this, yet its hardware efficiency is limited by the added prediction stage and the heavy memory traffic it entails. To address these limitations, this paper proposes BitStopper, a fine-grained algorithm-architecture co-design that operates without a sparsity predictor. First, a bit-serial enable stage fusion (BESF) mechanism is proposed to reuse and minimize the memory access by progressively terminating trivial tokens and merging the prediction stage into the execution stage. Second, a lightweight and adaptive token selection (LATS) strategy is developed to work in concert with the bit-level sparsity speculation. Third, a bit-level asynchronous processing (BAP) strategy is employed to improve compute utilization during the on-demand bit-grained memory fetching. Finally, an elaborate architecture is designed to translate the theoretical complexity reduction into practical performance improvement. Extensive evaluations demonstrate that, compared to state-of-the-art (SOTA) Transformer accelerators, BitStopper achieves 2.03x and 1.89x speedups over Sanger and SOFA, respectively, while delivering 2.4x and 2.1x improvements in energy efficiency.

Transformer注意力加速低功耗硬件协同

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。