提出高效量化计算核，显著提升4比特权值8比特激活大模型推理速度。
LiquidGEMM: Hardware-Efficient W4A8 GEMM Kernel for High-Performance LLM Serving
- 设计轻量级解量化方法与隐式细粒度流水线，避免硬件瓶颈。
- 相比现有方案最高提速2.90倍，系统级速度提升达4.94倍。
- 适合追求极致推理性能的AI服务部署场景。
量化是加速大语言模型推理的关键技术，能减少内存占用并提升计算效率。其中，4比特权值与8比特激活量化（W4A8）在精度与性能间取得良好平衡。然而，现有W4A8 GEMM内核因CUDA核心上解量化效率低下，无法跟上张量核心的高吞吐能力。本文提出LiquidGEMM，一种面向高性能大模型服务的硬件高效W4A8 GEMM内核。其核心包括：LiquidQuant，一种仅需每四元素两步算术指令即可实现快速且溢出安全解量化的方法；以及隐式细粒度流水线，可跨瓦片组完全重叠权重加载、解量化与矩阵乘法运算，无需软件同步或冗余内存访问。实验表明，LiquidGEMM相比当前最优的W4A8内核最高提升2.90倍，系统级速度最高达4.94倍。相较于NVIDIA TensorRT-LLM中多种量化GEMM内核，其性能提升1.12–1.63倍，系统级最高提速1.63倍。
原文摘要 · Abstract (English)
Quantization is a critical technique for accelerating LLM inference by reducing memory footprint and improving computational efficiency. Among various schemes, 4-bit weight and 8-bit activation quantization (W4A8) offers a strong balance between accuracy and performance. However, existing W4A8 GEMM kernels fall short in practice due to inefficient dequantization on CUDA Cores, which cannot keep pace with the high throughput of Tensor Cores. In this paper, we present LiquidGEMM, a hardware-efficient W4A8 GEMM kernel for efficient LLM serving. LiquidGEMM designs two key techniques: LiquidQuant, a hardware-efficient quantization method that enables fast, overflow-safe dequantization using just two arithmetic instructions per four elements; and an implicit fine-grained pipeline that fully overlaps weight loading, dequantization, and MMA across warp groups without software synchronization or redundant memory traffic. Experimental results show that LiquidGEMM achieves up to 2.90x speedup over state-of-the-art W4A8 kernels and up to 4.94x end-to-end system-level speedup. Compared to various quantized GEMM kernels in NVIDIA TensorRT-LLM, LiquidGEMM delivers 1.12-1.63x performance gains, and achieves up to 1.63x system-level speedup.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。