OASIS通过查表法实现非均匀量化矩阵乘，显著提升大模型推理速度与能效。
OASIS: Outlier-Aware LUT-Based GEMM with Dual-Side Quantization for LLM Inference Acceleration
- 用预计算的查表结构避免反量化开销，支持非均匀量化矩阵乘。
- 在保持仅1.98%精度损失下，比现有方案快3倍、能效高44%。
- 创新性地结合异常值检测与补偿机制，适合部署于边缘设备的大模型推理。
大语言模型（LLMs）在各类应用中表现出色，但推理时需大量内存与计算资源。现有量化方法存在效率与精度的权衡：仅权重量化（WOQ）带来高昂的反量化开销，而整数权重与激活量化（INT-WAQ）降低精度并损害模型质量。非均匀权重与激活量化（NU-WAQ）虽能更好捕捉分布不均特性，却难以适配传统低精度计算单元。本文提出OASIS，一种基于查找表（LUT）的架构，可在无需反量化的情况下高效实现非均匀量化权重与激活的通用矩阵乘（GEMM）。OASIS采用预计算的笛卡尔积查找表，使查表规模减小64倍，并将计算并行度提升至现有方法的1024倍。为在激进激活量化下保持精度，OASIS引入异常值感知量化方案，配合基于查表的矩阵乘与异常值误差补偿。此外，设计了Orizuru——一个高效的top-k检测引擎，用于实时识别激活异常值。大量实验表明，相较于FP16基线，OASIS平均精度下降仅1.98%，优于Atom的5.18%。硬件层面，相比FIGLUT加速器，平均提速3.00倍，能效提升1.44倍。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated impressive capabilities across a wide range of applications, but demand substantial memory and compute resources during inference. Existing quantization methods expose a trade-off between efficiency and accuracy: weight-only quantization (WOQ) incurs costly dequantization overheads, while integer weight-and-activation quantization (INT-WAQ) reduces precision and degrades model quality. Non-uniform weight-and-activation quantization (NU-WAQ) can better capture the non-uniform distributions of LLM weights and activations, yet remains incompatible with conventional low-precision compute units. This paper presents OASIS, a lookup table (LUT)-based architecture that enables efficient general matrix multiplication (GEMM) between non-uniformly quantized weights and activations without requiring dequantization. OASIS employs pre-computed Cartesian Product LUTs, achieving a 64x reduction in LUT size and enabling a 1024x higher computational parallelism over existing LUT-based GEMM methods. To preserve accuracy under aggressive activation quantization, OASIS introduces an outlier-aware quantization scheme with concurrent LUT-based GEMM and error compensation for outliers. Furthermore, we design Orizuru, an efficient top-k detection engine for real-time activation outlier identification. According to extensive evaluations, OASIS incurs an average accuracy drop of only 1.98% compared to the FP16 baseline, which is 5.18% lower than Atom. On the hardware side, OASIS achieves an average 3.00x speedup and a 1.44x energy efficiency improvement compared to the FIGLUT accelerator.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。