arXiv:2604.24820cs.ARcs.AI2026-04

专为长文本生成设计的高效推理加速器,显著提升速度与能效。

Salca: A Sparsity-Aware Hardware Accelerator for Efficient Long-Context Attention Decoding

论文配图:Salca: A Sparsity-Aware Hardware Accelerator for Efficient Long-Context Attention Decoding
图 1 · 摘自论文原文
  • 软硬件协同设计,采用稀疏注意力与低精度量化降低计算开销。
  • 实测比A100快3.82倍,能效高74.19倍,支持超长上下文推理。
  • 适合需要长文本处理的AI应用,如文档摘要、复杂问答系统。

长上下文可提升大模型能力,但带来严重硬件挑战:计算和内存开销随序列长度线性增长。尤其在解码阶段,持续访问庞大的键值缓存(KV cache),大幅增加带宽与算力压力。现有加速器主要针对短上下文设计,处理长上下文时性能急剧下降。为此,我们识别出核心瓶颈,提出一种面向长上下文注意力解码的软硬件协同加速器。软件层面,提出双压缩动态稀疏注意力,结合超低精度量化与特征稀疏性以最小化预测开销;硬件友好的近似Top-K选择将滤波复杂度从$O(n \log k)$降至$O(n)$。硬件层面,深度优化计算与内存访问,应对稀疏注意力与长上下文间的复杂交互,并建立性能模型以推导最优协同设计方案。最终硬件采用全流水并行架构,在长序列下仍实现$O(n)$效率。实验表明,该设计相较A100提速3.82倍,能效提升74.19倍;相比当前最先进加速器,是首个高效支持长上下文推理的ASIC,吞吐量至少提升3.5倍,能效提升2.08倍。

原文摘要 · Abstract (English)

Long contexts improve capabilities of large language models but pose serious hardware challenges: compute and memory footprints grow linearly with sequence length. Particularly, the decoding phase continuously accesses massive KV cache, dramatically increasing bandwidth and computing pressure. Existing accelerators are primarily designed and evaluated for short contexts. They suffer from significant performance degradation when processing long contexts. To bridge this gap, we identify the major bottleneck and present a hardware accelerator for long context attention decoding via hardware-software co-design. On the software side, we propose dual-compression dynamic sparse attention. It combines ultra-low-precision quantization with feature sparsity to minimize prediction overhead. A hardware-friendly approximate Top-K selection further reduces filter complexity from $O(n \log k)$ to $O(n)$. On the hardware side, we deeply optimize compute and memory access to tackle bottlenecks from intricate interplay between sparse attention and long contexts, and establish a performance model to derive the optimal co-design scheme. The resulting hardware adopts a fully pipelined parallel architecture and achieves $O(n)$ efficiency even for long sequences. Experiments show that our design delivers $3.82\times$ speedup and $74.19\times$ energy efficiency over A100. Compared to SOTA accelerators, this is the first ASIC accelerator that efficiently supports long context inference, with at least $3.5\times$ higher throughput and $2.08\times$ better energy efficiency.

长上下文硬件加速稀疏注意力能效优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。