arXiv:2412.10059cs.ARcs.AI2024-12中稿 · HPCA 2025被引 8

提出新型神经网络加速器,兼顾高精度与低功耗。

Panacea: Novel DNN Accelerator using Accuracy-Preserving Asymmetric Quantization and Energy-Saving Bit-Slice Sparsity

  • 采用非对称量化+比特切片稀疏压缩,提升精度
  • 新架构支持非零切片压缩,降低能耗35%以上
  • 适合大规模推理场景的硬件优化设计

低比特精度及其比特切片稀疏性被用于加速大规模深度神经网络(DNN)推理中的通用矩阵乘法(GEMM)。传统对称量化虽支持权重与激活的切片稀疏性,但因激活分布不对称导致精度损失,难以接受。为缓解此问题,近年研究采用非对称量化激活,无需额外操作。然而,现有非对称量化产生大量非零切片,无法被当前切片级GEMM加速器压缩跳过,反而增加处理能耗。为此,本文首次提出非对称量化比特切片GEMM(AQS-GEMM),不仅跳过零切片,还压缩频繁出现的非零切片并跳过其运算。为提升激活的切片级稀疏性,提出两种软硬件协同优化方法:零点调控与基于分布的比特切片。为支持上述设计,本文提出全新DNN加速器Panacea,高效处理分块AQS-GEMM的稀疏/密集负载,通过专用数据流和游程编码最大化数据复用、最小化外部内存访问,显著提升硬件效率。基准测试表明,Panacea优于现有主流DNN加速器。

原文摘要 · Abstract (English)

Low bit-precisions and their bit-slice sparsity have recently been studied to accelerate general matrix-multiplications (GEMM) during large-scale deep neural network (DNN) inferences. While the conventional symmetric quantization facilitates low-resolution processing with bit-slice sparsity for both weight and activation, its accuracy loss caused by the activation's asymmetric distributions cannot be acceptable, especially for large-scale DNNs. In efforts to mitigate this accuracy loss, recent studies have actively utilized asymmetric quantization for activations without requiring additional operations. However, the cutting-edge asymmetric quantization produces numerous nonzero slices that cannot be compressed and skipped by recent bit-slice GEMM accelerators, naturally consuming more processing energy to handle the quantized DNN models. To simultaneously achieve high accuracy and hardware efficiency for large-scale DNN inferences, this paper proposes an Asymmetrically-Quantized bit-Slice GEMM (AQS-GEMM) for the first time. In contrast to the previous bit-slice computing, which only skips operations of zero slices, the AQS-GEMM compresses frequent nonzero slices, generated by asymmetric quantization, and skips their operations. To increase the slice-level sparsity of activations, we also introduce two algorithm-hardware co-optimization methods: a zero-point manipulation and a distribution-based bit-slicing. To support the proposed AQS-GEMM and optimizations at the hardware-level, we newly introduce a DNN accelerator, Panacea, which efficiently handles sparse/dense workloads of the tiled AQS-GEMM to increase data reuse and utilization. Panacea supports a specialized dataflow and run-length encoding to maximize data reuse and minimize external memory accesses, significantly improving its hardware efficiency. Our benchmark evaluations show Panacea outperforms existing DNN accelerators.

DNN加速器量化稀疏性硬件优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。