针对大模型部署难题，提出感知权重密度的量化方法
DAQ: Density-Aware Post-Training Weight-Only Quantization For LLMs
- 按权重密度分布调整量化范围，对齐高密度区域
- 动态调整缩放因子和零点，降低困惑度22.8%以上
- 适合需要高效部署大模型的工程师和研究者
大型语言模型在多项任务中表现优异，但受限于硬件条件难以部署。本文提出密度感知的后训练权重量化方法（DAQ），包含两个阶段：1）以密度为中心的对齐，识别高密度权重中心，并将动态范围对准该点，使高密度权重区域与浮点高精度区域对齐；2）可学习的动态范围调节，通过优化量化参数（即缩放因子和零点）来最小化权重对模型输出的影响。在LLaMA和LLaMA-2上的实验表明，DAQ持续优于最佳基线方法，平均将困惑度损失降低22.8%（LLaMA）和19.6%（LLaMA-2）。代码已开源：https://github.com/LuoYingSong/DAQ。
原文摘要 · Abstract (English)
Large language models (LLMs) excel in various tasks but face deployment challenges due to hardware constraints. We propose density-aware post-training weight-only quantization (DAQ), which has two stages: 1) density-centric alignment, which identifies the center of high-density weights and centers the dynamic range on this point to align high-density weight regions with floating-point high-precision regions; 2) learnable dynamic range adjustment, which adjusts the dynamic range by optimizing quantization parameters (i.e., scale and zero-point) based on the impact of weights on the model output. Experiments on LLaMA and LLaMA-2 show that DAQ consistently outperforms the best baseline method, reducing perplexity loss by an average of 22.8% on LLaMA and 19.6% on LLaMA-2. Our code is available at https://github.com/LuoYingSong/DAQ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。