让模型解释更可信：用可读编码确保说出的内容真实可查。
Train the Model, Not the Reader: Decodability Supervision for Verifiable Activation Explanations
- 训练辅助头保证关键信息可被探测，避免模型用虚假表述骗过评估
- 传统方法下解释虽重建效果好，但多数内容不依赖具体事实
- 适合关注模型可解释性与安全性的研究者，尤其重视真相验证场景
自然语言自编码器通过重构效果评估隐藏激活的解释是否忠实。若激活能从解释中重建，则认为解释可信。然而该测试对单个错误陈述不敏感：即使修改某句话也不影响重构，该说法就不会被惩罚。我们发现此测试可通过两种非忠实方式通过：在释放的 Qwen-2.5-7B 语言化器上，解释重构得分远超随机水平，但仅约 2% 的具体陈述影响重构结果，说明评分反映的是大意而非事实细节。在精确合成真实数据下，标准训练始终产生共适应的私有编码（即重构依赖的虚假表述），且不影响目标模型的修正也无效。本文提出两种审计协议——对比接地与真相、交换至独立评估者，并引入 RECAP（通过协同训练辅助预测器实现可读编码）。在 RECAP 训练的沙盒模型中，新语言化器真正传达指定内容，编码消失，代价仅为 +0.001 nat。该结果在预训练的 Pythia-160M 上复现。探测器可稳定解码内容，但新语言化器仅部分传达（真值 0.44–0.46 对比接近零的对照组）。对于可解释性，高重构率不证明个别声明为真；对于 AI 安全，RECAP 使指定内容可由探测器验证，而非依赖模型自我陈述。独立探测器将真实陈述排在虚假之上（AUC 0.96 vs 0.82 无 RECAP）。面对篡改解释以最大化分数但隐瞒约 87% 欺诈成本的对手，RECAP 探测器仍能识别谎言（AUC 0.95），而对照探测器退化至随机水平（0.51）。
原文摘要 · Abstract (English)
Natural-language autoencoders score explanations of hidden activations by reconstruction. An explanation is deemed faithful if the activation can be regenerated from it. The test is structurally insensitive to individual false claims. If flipping a claim does not change the reconstruction, the claim is never penalized. We show the test is passed in two ways, neither faithful. On a released Qwen-2.5-7B verbalizer, explanations reconstruct well above chance while ~2% of specific claims are ones the reconstruction depends on, so the score tracks gist, not specific facts. Under exact synthetic ground truth, standard training consistently develops co-adapted private codes (false wording the reconstruction depends on), and fixes that leave the target model unchanged do not help. We contribute two audit protocols, the comparison of grounding and truth and the swap to an independent evaluator, and RECAP (Readable Encodings via Co-trained Auxiliary Predictors), linear heads trained alongside the target model to keep designated content decodable. On RECAP-trained sandbox models, fresh verbalizers state the designated content truly and the codes vanish, at a +0.001-nat cost. This replicates on a pretrained Pythia-160M. The content becomes reliably decodable by a probe, though a fresh verbalizer conveys it only in part (truth 0.44-0.46 vs a near-zero control). For interpretability, high reconstruction does not certify individual claims. For AI safety, RECAP makes designated content checkable against a probe rather than asserted by prose a model can game. An independent probe ranks the verbalizer's true claims above its false ones (AUC 0.96 vs 0.82 without RECAP). Against an adversary that edits an explanation to maximize the score while lying (suppressing ~87% of its lie penalty), the RECAP probe still flags the lies (AUC 0.95) while the control probe collapses to chance (0.51).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。