arXiv:2604.03298cs.ARcs.DC2026-04中稿 · ISCA 2026, 17 page…被引 2

针对昇腾芯片优化的无损模型压缩技术,显著提升大模型推理速度。

ENEC: A Lossless AI Model Compression Method Enabling Fast Inference on Ascend NPUs

论文配图:ENEC: A Lossless AI Model Compression Method Enabling Fast Inference on Ascend NPUs
图 1 · 摘自论文原文
  • 基于分块定长编码,结合昇腾特性进行比特打包与向量化计算
  • 压缩比优于nvCOMP 1.12倍,推理吞吐达DietGPU的3.43倍
  • 首个在昇腾上性能媲美顶级GPU压缩方案的开源无损压缩工具

大语言模型的快速扩展给部署与推理带来挑战,尤其在华为昇腾等资源受限的专用AI加速器上,权重数据传输已成为关键性能瓶颈。尽管无损压缩可保持模型精度并减少数据量,但现有算法在昇腾NPU上运行时吞吐极低。本文提出ENEC,一种专为昇腾神经处理单元优化的模型权重无损压缩方法。ENEC采用分块定长编码,结合一系列NPU特性优化:分层比特减半打包、向量化无分支整数变换、解耦依赖的段内前缀和计算。实验表明,ENEC在压缩比和吞吐量上均超越现有顶尖NPU压缩器。相比领先GPU方案,其吞吐量达DietGPU的3.43倍,压缩比优于nvCOMP 1.12倍。通过降低权重传输开销,显著提升端到端推理性能,最高实现6.3倍加速。在昇腾平台,ENEC是首个性能接近顶级GPU压缩器的开源无损压缩算法,为大规模AI模型部署提供有效解决方案。

原文摘要 · Abstract (English)

The rapid scaling of Large Language Models presents significant challenges for their deployment and inference, particularly on resource-constrained specialized AI hardware accelerators such as Huawei's Ascend NPUs, where weight data transfer has become a critical performance bottleneck. While lossless compression can preserve model accuracy and reduce data volume, existing lossless compression algorithms exhibit extremely low throughput when ported to the Ascend NPU architecture. In this paper, we propose ENEC, a novel lossless compression method specifically customized for AI model weights and optimized for Ascend Neural Processing Units. ENEC adopts a block-based fixed-length encoding scheme and incorporates a series of NPU-specific optimizations: bit-width quantization with hierarchical halving bit-packing, vectorized branch-free integer transformation, and dependency-decoupled intra-segment scan for efficient prefix-sum computation. Experimental results demonstrate that ENEC outperforms existing state-of-the-art NPU compressors in both compression ratio and throughput. Compared to leading GPU solutions, ENEC achieves a 3.43X higher throughput than DietGPU and a 1.12X better compression ratio than nvCOMP. By reducing weight transmission overhead, ENEC significantly improves end-to-end inference performance, achieving up to a 6.3X speedup. On Ascend NPUs, ENEC is the first open-source lossless compression algorithm for model weights that achieves performance comparable to state-of-the-art GPU compressors, offering an effective solution for deploying large-scale AI models.

模型压缩昇腾NPU无损压缩推理加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。