arXiv:2607.25504cs.ARcs.AI2026-07

为向量处理器设计稀疏张量运算单元,显著提升Transformer推理速度。

At-the-Roofline Sparse Tensor Contractions on Vector Processors for Transformer Inference

论文配图:At-the-Roofline Sparse Tensor Contractions on Vector Processors for Transformer Inference
图 1 · 摘自论文原文
  • 通过索引聚集-累加-散列支持,实现稀疏张量收缩的高效执行
  • 在40%-60%双重稀疏下,推理速度比密集基线快2.4到5.25倍
  • 仅3.1%面积开销,适合大规模稀疏模型部署

细粒度权值剪枝与激活稀疏化已成为降低Transformer模型推理计算与内存开销的有效方法。在中等稀疏度下,Gustavson数据流天然适配向量处理器上同时利用激活与权值稀疏性的执行模式,但现有RVV架构缺乏原生支持,导致内核依赖软件索引解码与L1背靠的索引内存操作,使稀疏张量收缩远低于其屋顶性能。本文提出Ventaglio,一种可配置的稀疏执行单元,配合RVV指令集扩展,通过索引聚集-累加-散列支持推动稀疏张量收缩逼近其屋顶性能。集成于开源向量处理集群,基于12nm FinFET实现,相比优化后的RVV基线加速6.9–7.4倍,单集群仅增加3.1%面积开销。构建了高精度指令级性能模型,经寄存器传输级实现校准,用于大规模4×4多集群系统扩展分析。使用双稀疏度达40–60%的DuoGPT剪枝版LLaMA-3-8B模型,在预填充与自回归解码阶段分别获得2.40–5.25×和2.06–3.16×的加速优势。

原文摘要 · Abstract (English)

Fine-grained weight pruning and activation sparsification have emerged as effective approaches for reducing the compute and memory cost of inference for Transformer models. In the moderate-sparsity regime, Gustavson's dataflow provides a natural execution model for exploiting both activation and weight sparsity on vector processors through metadata-driven indexed accumulation. However, existing RVV architectures lack native support for this pattern, forcing kernels to rely on software index decoding and L1-backed indexed memory operations that keep sparse tensor contractions far below their roofline performance bound. We present Ventaglio, a runtime-configurable sparse execution unit coupled with RVV ISA extensions that drives sparse tensor contractions toward their roofline through indexed gather-accumulate-scatter support. Integrated into an open-source vector processing cluster and implemented in 12nm FinFET, Ventaglio accelerates sparse tensor contraction kernels by $6.9\text{--}7.4\times$ over optimized RVV baselines, with only $3.1\%$ area overhead for a cluster of tightly-L1 coupled vector processing elements. We build a performance-accurate instruction-level model of the Ventaglio extension, calibrate it against RTL implementation, and leverage it for scale-out performance analysis on a large $4\times4$ multi-cluster system. Using a DuoGPT-pruned LLaMA-3-8B model with practical $40\text{--}60\%$ dual sparsity, Ventaglio achieves $2.40\text{--}5.25\times$ and $2.06\text{--}3.16\times$ speedup over dense baselines during prefill and autoregressive decoding, respectively.

稀疏计算向量处理器Transformer硬件加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。