量化让小模型更单一，大模型变啰嗦，但不增加偏见。
Quantization Amplifies Determinism, Not Bias: Scale-Dependent Behavioral Effects of Serving-Time Weight Compression
- 在相同条件下测试不同精度的模型，发现4位量化降低小模型输出多样性。
- 8B模型推荐一致率上升5.1个百分点，大模型则出现语气符号增多等风格变化。
- 提示词和种子相同的成对生成数据，揭示量化对行为的非线性影响。
权重量化显著影响开源大模型的服务经济性。通常以能力基准评估其成本，中等规模模型4比特量化常被视为“几乎无代价”。本文提出新问题：当多个答案合理时，量化是否改变模型的选择？我们固定硬件、软件与采样配置，对Qwen3-8B/14B/32B三个检查点，在W4A16 AWQ、W8A16 FP8-Marlin和bf16三种精度下服务，并收集约7.1万条配对完成（同提示词与随机种子），来自两个自定义且防泄露的提示电池。分析按版本控制预设三波进行。在8B模型上，整数4比特使输出多样性下降：同一场景下两样本推荐同一品牌概率提升5.1个百分点（提示配对符号翻转检验，Holm p = .023；全重生成验证为+4.4个百分点）；词汇多样性显著下降（TTR -0.011，标准化效应量 -0.51；在长度控制测量下仍稳健）。在14B与32B模型上，内容集中度指标均不显著，取而代之的是风格漂移：14B模型每千字破折号使用增加0.46，32B增加0.61（两者Holm p ≤ .0024）。对刻板印象方向的预设检验均为零结果：输出集中在每个提示的主流答案，而非刻板答案。机制上，令牌级分布趋于平坦（决策令牌熵 +0.091比特，p = .015），而语义分布（通过首令牌对数概率直接衡量）变得更集中（碰撞率 +2.6个百分点，p = .023）——即单个词更不可预测，但整体含义更重复。在最小的8B模型上，AWQ-int4服务明显缩小建议范围；审计应评估集中度而非仅偏见。
原文摘要 · Abstract (English)
Weight quantization largely determines the economics of serving open-weight LLMs. Its costs are usually assessed with capability benchmarks, on which 4-bit quantization of mid-sized models is often considered "nearly free." We examine a different question: when several answers are valid, does quantization change what a model chooses to say? We serve three checkpoints (Qwen3-8B/14B/32B) at three weight precisions (W4A16 AWQ, W8A16 FP8-Marlin, and bf16), holding the hardware, software, and sampling configuration constant, and collect approximately 71,000 completions paired by prompt and seed across two custom, leak-checked prompt batteries. We pre-specified the analyses in three waves in version control. At 8B, int4 reduces output diversity: the probability that two samples for the same scenario recommend the same brand increases by 5.1 percentage points (prompt-paired sign-flip test, Holm p = .023; reproduced at +4.4pp on a full regeneration of the arm), and lexical diversity falls substantially (TTR -0.011, standardized effect -0.51; robust to a length-controlled measure). At 14B and 32B, no content-concentration measure reaches significance; instead, stylistic drift emerges (em-dash rate +0.46/1k words at 14B and +0.61/1k at 32B, both Holm p <= .0024). Pre-specified tests of stereotype direction are null at every scale: outputs concentrate on the modal answer for each prompt rather than on stereotypical answers. Mechanistically, the token-level distribution becomes flatter (decision-token entropy +0.091 bits, p = .015) while the semantic distribution, measured directly from first-token log probabilities, becomes more concentrated (collision +2.6pp, p = .023): individual tokens become less predictable even as meanings become more repetitive. At 8B, the smallest size tested, AWQ-int4 serving measurably narrows the range of suggestions; audits should assess concentration as well as bias.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。