量化后大模型行为控制仍有效，但长推理会崩溃，短推理则突然失效。
Steering Under Compression: Dose-Response, Capability Cost, and Failure Asymmetry in Quantized LLMs
- 用等效效果框架对比量化前后模型的行为控制成本
- 情感控制在量化后基本不变，推理长度控制却出现不对称崩溃
- 量化会改变模型基础能力，但控制方向仍与原模型高度一致
推理时激活值调节可无参数修改地控制大语言模型行为，而训练后量化能降低部署的内存和计算开销。尽管两者在实践中日益融合，其交互关系尚未明确。我们系统研究了在权重仅量化（INT8 和 NF4）下四种 7-9B 开源模型对情感判断和无评分推理长度的控制效果。采用等效效果框架比较相同行为效应下的能力代价，发现情感调节在量化后保持稳定：校正 GSM8K 解析器缺陷后，合并 INT8 对比为 -0.010（90% 置信区间 [-0.026, +0.007]），符合预注册三标签规则下的描述性等效；而 NF4 结果不明确，为 -0.017（[-0.067, +0.033]）。相反，推理长度呈现意外的非对称剂量反应：延长是渐进式的，但会因上限溢出导致崩溃；缩短则为阶跃函数，仅能实现 12-30% 的缩短（取决于模型），随后发生突变失败。我们揭示了一种方法论陷阱：传统等效阶梯锚定于崩溃底端，对有下限的目标造成误判，并提出截尾构造以恢复可解释的交叉点。此外，我们量化发现 Mistral-NF4 在未启用调控时（alpha=0）GSM8K 得分从 0.545 降至 0.365，表明压缩本身即可主导调控干预。尽管如此，调控向量与原始浮点版本仍高度共线（INT8 为 0.989–0.998，NF4 为 0.945–0.990），证实行为方向在量化后仍得以保留，即使代价结构已改变。所有代码与数据均已公开。
原文摘要 · Abstract (English)
Inference-time activation steering enables behavioral control of large language models without parameter modification, while post-training quantization reduces memory and compute costs for deployment. Despite their growing convergence in practice, the interaction between these two techniques remains uncharacterized. We systematically study activation steering under weight-only quantization (INT8 and NF4) across four open-weight 7-9B models and two behavioral targets: judged sentiment and judge-free reasoning length. Using an iso-effect framework that compares capability costs at matched behavioral effect, we find that sentiment steering survives quantization intact. After correcting a GSM8K parser artifact with a uniform v2.3.1 rescore, the pooled INT8 contrast is -0.010 (90% CI [-0.026, +0.007]), descriptively Equivalent under the preregistered three-label rule, while NF4 remains Inconclusive at -0.017 ([-0.067, +0.033]). In contrast, reasoning length exhibits a surprising asymmetric dose-response: lengthening is graded but terminates in cap-runaway and collapse, while shortening is a step function with only 12-30% shortening (model-dependent) before discontinuous failure. We expose a methodological pitfall: the naive iso-effect ladder anchors on the collapse floor for floor-bounded targets, and we introduce a censored construction that restores interpretable crossings. We also quantify a substantial baseline capability shift for Mistral-NF4 (0.545 to 0.365 GSM8K at alpha=0), demonstrating that compression can dominate the steering intervention. Despite this, steering vectors remain highly collinear with their FP16 siblings (cosine similarity 0.989-0.998 for INT8, 0.945-0.990 for NF4), confirming that the behavioral direction survives quantization even when the cost structure does not. All code and data are released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。