量化后模型质量不能代表安全，很多低安全风险模型仍表现良好。
Quality Is Not a Safety Proxy Under Quantization
- 通过6个模型、7级量化层级测试，发现质量与安全方向常不一致。
- 9个隐藏危险样本质量稳定甚至提升，拒答率下降12%-68%。
- 提出新评估指标RTSI，可精准识别高危量化模型，适合安全审查者使用。
量化检查点通常先用质量指标筛选，再进行安全测试。本文在包含6个模型、4个模型家族、7级GGUF量化层级及AWQ/GPTQ INT4检查点的51行矩阵中检验该方法。结果表明：所有36组质量-安全配对的方向均不一致；其中9个隐藏危险行和1个接近危险行的质量保持或改善，但拒答率下降12%-68个百分点；11个危险行中有7个来自AWQ/GPTQ。对17个基于Hugging Face的FP16/AWQ/GPTQ单元的四探针机制分析显示，熵、拒答方向和校准探针无法有效区分危险行；尽管安全相关神经元吸收的量化误差高出1.39倍（p < 5×10⁻⁷），但该现象无模型架构特异性。Claude Sonnet 4在预定义分层集上重标11,470项，与主判官gemma3:12b一致率达89.9%（κ=0.873，95% CI [0.866, 0.881]），且未改变任何危险行。基于四个拒答模板漂移特征构建的内部行为评估工具RTSI，可将10/10隐藏或近隐藏危险行引导至直接安全测试（威尔逊95%置信下界0.72），同时使45个非基线行中23个保留在低风险桶内，且在样本内评分和逐行留一验证中均有效；最佳单特征基线（唯一前缀率差值、原始拒答率差值）在相同桶大小下分别召回9/10和8/10，跨堆栈迁移需重新校准。研究结论：在所考察的量化检查点、模型家族和安全结果下，保留的质量无法替代直接安全评估。
原文摘要 · Abstract (English)
Quantized checkpoints are often screened first with quality metrics and only later, if at all, with direct safety tests. This paper audits that shortcut on a matched 51-row matrix spanning 6 models, 4 families, a 7-level GGUF ladder, and AWQ/GPTQ INT4 checkpoints. In this matrix the shortcut fails: all 36 quality-safety pairings split direction across models, and 9 hidden-danger rows plus 1 near-hidden-danger row show quality stable or improved while refusal falls by 12-68 percentage points. Seven of the 11 AWQ/GPTQ rows are hidden-danger. A four-probe mechanistic follow-up over the 17 Hugging Face-backed FP16/AWQ/GPTQ cells does not rescue it: entropy, refusal-direction, and calibration probes are weak or null separators of dangerous rows, and although probe-identified safety-associated neurons absorb 1.39$\times$ more quantization error overall ($p < 5 \times 10^{-7}$), the effect is not regime-specific. Claude Sonnet 4 relabels 11,470 items in a predefined stratified set, agrees with the primary gemma3:12b judge on 89.9\% of rows ($κ= 0.873$, 95\% CI [0.866, 0.881]), and changes 0/10 hidden-danger cells. A calibrated study-internal behavioral screen -- the Refusal Template Stability Index (RTSI), built from four refusal-template drift features and calibrated on this matrix -- routes 10/10 hidden- or near-hidden-danger rows to direct safety testing (Wilson 95\% CI lower bound 0.72) while leaving 23 of 45 non-baseline rows in a low-risk bucket under both in-sample scoring and row-level leave-one-out validation; on the same matrix, the best single-feature baselines (unique-prefix-rate-delta, raw refusal-rate delta) recover 9/10 and 8/10 respectively at matched bucket size, and cross-stack transfer requires recalibration. For the quantized checkpoints, model families, and safety outcomes studied here, retained quality cannot waive direct safety evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。