小模型修复大模型在敏感话题上的压制问题，不改主干也能提升事实准确性。
Correcting Suppressed Log-Probabilities in Language Models with Post-Transformer Adapters
- 用0.02%参数的后置适配器，基于冻结隐藏状态修正敏感话题概率抑制。
- 在31个意识形态相关事实中恢复准确度，泛化到11%-39%未见事实，零知识退化下仍有效。
- 仅在当前词预测位置应用适配器可生成连贯文本，证明隐藏层干预更优。
对齐优化的语言模型常在政治敏感话题上压制真实信息的对数概率，尽管其隐藏表示仍保留知识。我们展示一个786K参数（约基座模型的0.02%）的后变压器适配器，在冻结的隐藏状态上训练，能纠正Qwen3-4B、8B和14B在31个意识形态判别性事实上的概率抑制。适配器记忆全部15个训练事实，并在每种规模下5次随机划分中泛化至11%-39%的16个未见事实，且通过锚定训练实现零知识退化。门控（SwiGLU）与无门控（线性瓶颈）适配器表现相当（所有规模下Fisher精确检验p > 0.09）。在指令模型上，适配器可纠正对数概率排序。若在整个生成过程中应用，适配器产生不连贯输出；但仅在当前预测位置应用（最后位置仅用），则生成连贯且较少审查的内容。在投影后对数空间的适配器无论何种应用方式均无法生成连贯文本，表明生成修正应在隐藏状态层面进行。此前未记录的苹果MLX中一个静默梯度缺陷解释了早期所有无效结果：标准写法nn.value_and_grad(model, fn)(model.parameters())返回零梯度而无报错；正确写法nn.value_and_grad(model, fn)(model, data)可解决此问题。我们提供最小复现代码并讨论对其他使用MLX的适配器研究的影响。
原文摘要 · Abstract (English)
Alignment-tuned language models frequently suppress factual log-probabilities on politically sensitive topics despite retaining the knowledge in their hidden representations. We show that a 786K-parameter (approximately 0.02% of the base model) post-transformer adapter, trained on frozen hidden states, corrects this suppression on 31 ideology-discriminating facts across Qwen3-4B, 8B, and 14B. The adapter memorizes all 15 training facts and generalizes to 11--39% of 16 held-out facts across 5 random splits per scale, with zero knowledge regressions via anchored training. Both gated (SwiGLU) and ungated (linear bottleneck) adapters achieve comparable results; neither consistently outperforms the other (Fisher exact p > 0.09 at all scales). On instruct models, the adapter corrects log-probability rankings. When applied at all token positions during generation, the adapter produces incoherent output; however, when applied only at the current prediction position (last-position-only), the adapter produces coherent, less censored text. A logit-space adapter operating after token projection fails to produce coherent generation at any application mode, suggesting hidden-state intervention is the correct level for generation correction. A previously undocumented silent gradient bug in Apple MLX explains all null results in earlier iterations of this work: the standard pattern nn.value_and_grad(model, fn)(model.parameters()) returns zero gradients without error; the correct pattern nn.value_and_grad(model, fn)(model, data) resolves this. We provide a minimal reproduction and discuss implications for other adapter research using MLX.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。