低比特量化会削弱模型决策信心，导致安全拒绝能力下降。
Which Decisions Low-Bit Quantization Breaks, and How to Predict Them
- 通过测量决策置信度（边际）变化，发现量化损伤随位宽线性收缩。
- 3比特时工具调用决策倾向保守，2比特时安全拒绝率几乎归零。
- 可精准预测哪些决策会失效，适合部署前评估压缩模型可靠性。
量化在低于4比特时会损害模型性能，但现有方法无法判断具体哪些决策会改变。本文通过测量决策边际（所选选项得分与次优选项之差）在量化前后的变化，发现损伤是比例性的：边际被乘以一个随位宽下降的因子（4比特中位数0.86，3比特降至0.33，2比特为0.00），称为边际收缩。该机制解释了为何高置信决策在低比特下仍可能出错，且方向由模型自身偏好决定——3比特时工具调用倾向于取消，而工具选择本身不变。任何基于固定噪声或增长噪声的假设均无法解释全部378个受损的‘是否调用’或安全相关决策。使用每模型每比特的边际数据集可准确预测未见决策的翻转率，中位误差仅1.7点，校准后对161,744次预测误差仅为0.004。借用他人常数则误差达18-33点，故必须针对每模型每比特实测。4比特时以最可能词锚定行为（85%工具项中正确选项为两个候选之一），2比特视为测量下限。测试的所有无标签修复方法均无法恢复超过1比特的性能。
原文摘要 · Abstract (English)
Quantization is known to hurt below four bits, but nobody can say which of a model's decisions will change at a given bit-width. This matters most where a model acts rather than answers: a compressed agent stops calling its tools and, one bit lower, loses roughly half its safety refusals, while benchmark scores barely move. Prior work assumes the added noise has a roughly fixed size, which would make confident decisions safe. We measure the decision instead: the margin, the picked option's score minus its best alternative's, tracked before and after quantization across 16 models from 8 families under round-to-nearest, seven under AWQ, two under GPTQ and one under GGUF, at 8 down to 2 bits. The damage is proportional, not fixed in size: the margin is multiplied by a factor that collapses with bit-width (median 0.86 at 4 bits, 0.33 at 3, 0.00 at 2), which we call margin shrinkage. Contraction removes the protection a large margin affords; the model's own biases pick the direction: at 3 bits the decision to call a tool collapses toward inaction while the choice of which tool is untouched. No additive account, including one whose noise grows with the margin, wins a single damaged whether-to-call or safety cell (378 of 378). Given a condition's own constants the relation predicts held-out flip rates to a median 1.7 points, calibrated per decision (error 0.004 over 161,744 predictions), no flip used in the fit. Borrowed constants are wrong by 18-33 points at 3 bits, so the paired margin set has to be measured per model and bit-width: it locates breaking decisions without replacing measurement. At 4 bits the measurement is anchored to behaviour (the most likely token over the whole vocabulary is one of the item's two options in 85% of tool items); we treat the 2-bit floor as where the instrument stops measuring. No label-free repair we tested recovers more than one more bit does.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。