用新型压缩技术让大模型推理更快更省内存。
ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compression
- 设计固定长度编码与融合解压计算核，适配GPU架构
- 模型体积减30%，内核级速度提升2.21倍，端到端提速1.22倍
- 适合追求高速低存的LLM部署场景
无损模型压缩有望缓解比特精确大语言模型服务中的内存与带宽瓶颈。然而，现有方法因与GPU架构设计不匹配，常导致显著推理延迟：在核层面，传统熵编码产生的可变长度比特流破坏SIMT并行性；在系统层面，解耦流水线引发冗余内存访问。本文提出ZipServ，一种为高效大模型推理协同设计的无损压缩框架。ZipServ引入面向张量核心的三重位图编码（TCA-TBE），采用固定长度格式实现常数时间并行解码，并设计融合解压-矩阵乘法（ZipGEMM）核，在线将压缩权重直接解压至张量核心寄存器。该‘加载压缩数据、计算解压后结果’的设计消除中间缓冲区，最大化计算强度。实验表明，ZipServ使模型尺寸减少最高达30%，相比NVIDIA cuBLAS实现最高2.21倍的内核级加速，相比vLLM平均提升1.22倍的端到端推理速度。ZipServ是首个在GPU上同时实现存储节省与显著加速的无损压缩系统。
原文摘要 · Abstract (English)
Lossless model compression holds tremendous promise for alleviating the memory and bandwidth bottlenecks in bit-exact Large Language Model (LLM) serving. However, existing approaches often result in substantial inference slowdowns due to fundamental design mismatches with GPU architectures: at the kernel level, variable-length bitstreams produced by traditional entropy codecs break SIMT parallelism; at the system level, decoupled pipelines lead to redundant memory traffic. We present ZipServ, a lossless compression framework co-designed for efficient LLM inference. ZipServ introduces Tensor-Core-Aware Triple Bitmap Encoding (TCA-TBE), a novel fixed-length format that enables constant-time, parallel decoding, together with a fused decompression-GEMM (ZipGEMM) kernel that decompresses weights on-the-fly directly into Tensor Core registers. This "load-compressed, compute-decompressed" design eliminates intermediate buffers and maximizes compute intensity. Experiments show that ZipServ reduces the model size by up to 30%, achieves up to 2.21x kernel-level speedup over NVIDIA's cuBLAS, and expedites end-to-end inference by an average of 1.22x over vLLM. ZipServ is the first lossless compression system that provides both storage savings and substantial acceleration for LLM inference on GPUs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。