量化会损害大模型安全，需针对性优化压缩策略。
Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Models
- 构建1.067个高挑战性安全问题数据集，评估量化对模型行为影响。
- 66种量化变体显示：量化普遍降低安全对齐，无通用最优方法。
- 特定精度的量化方法（如4比特用QUIK）表现更优，应按精度选策略。
大语言模型虽强大，但计算开销限制其应用。量化可提升效率，但其对安全性与可信度的影响尚不明确。为此，我们提出OpenMiniSafety，一个由人工标注的安全数据集，包含1.067个高挑战性问题，用于严格评估模型行为。我们公开了四个大模型（全精度与量化版）的人工安全评估结果，共4.268组标注问答对。通过在四个安全基准上评估66种量化变体（四种后训练量化PTQ和两种量化感知训练QAT方法），发现无论是PTQ还是QAT均会削弱安全对齐，且QLORA或STE等QAT方法安全性更低。不同方法在各基准、精度设置和模型间无一致优势，表明需采用安全感知的压缩策略。此外，针对特定精度的方法（如4比特用QUIK、AWQ；2比特用AQLM、Q-PET）在其目标精度下表现更佳，说明这些方法并非通用压缩手段，而是适配特定精度的差异化方案。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are powerful tools for modern applications, but their computational demands limit accessibility. Quantization offers efficiency gains, yet its impact on safety and trustworthiness remains poorly understood. To address this, we introduce OpenMiniSafety, a human-curated safety dataset with 1.067 challenging questions to rigorously evaluate model behavior. We publicly release human safety evaluations for four LLMs (both quantized and full-precision), totaling 4.268 annotated question-answer pairs. By assessing 66 quantized variants of these models using four post-training quantization (PTQ) and two quantization-aware training (QAT) methods across four safety benchmarks including human-centric evaluations we uncover critical safety performance trade-offs. Our results show both PTQ and QAT can degrade safety alignment, with QAT techniques like QLORA or STE performing less safely. No single method consistently outperforms others across benchmarks, precision settings, or models, highlighting the need for safety-aware compression strategies. Furthermore, precision-specialized methods (e.g., QUIK and AWQ for 4-bit, AQLM and Q-PET for 2-bit) excel at their target precision, meaning that these methods are not better at compressing but rather different approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。