多智能体系统在同伴质疑下出错，主因并非对齐训练，而是中间层注意力机制被干扰。
Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy
- 发现错误响应源于模型中间层注意力异常，而非对齐导致的讨好行为
- 单一正确异议者可降低54-73个百分点的出错率
- 应从系统结构上设计分歧机制，而非依赖提示词防御
基于大模型的多智能体流水线在模拟同伴意见分歧时会出现错误答案，这种现象称为‘产量’（yield），通常归因于强化学习人类反馈（RLHF）引发的讨好倾向。我们测试了四种模型家族，发现预训练基础模型与指令微调版本表现出相同的替换模式，且基础模型的平均产量更高。通过激活拼接技术，定位到错误源自一个狭窄的中层窗口，该处注意力机制起主导作用，而MLP贡献可忽略；在此窗口之上进行拼接，可恢复96%的原始正确率与受压正确率之间的差距。攻击面分解为两个独立因素（通道框架和共识强度），其交互导致多数共识下出现47.5个百分点的产量差异，且在裁判数量N ∈ {4, 5, 6}范围内保持稳定。两种收敛的激活空间干预表明，压力抑制的是干净推理特征，而非激活新的讨好回路。单一正确反驳者在所有测试框架下将产量降低54至73个百分点，而最强的提示级防御在非设计攻击变体上失效。因此，缓解策略应针对该机制，通过流水线层级的结构化异议设计，而非依赖提示级防御。
原文摘要 · Abstract (English)
LLM-based multi-agent pipelines flip from correct to incorrect answers under simulated peer disagreement at rates we term yield, a vulnerability widely attributed to RLHF-induced sycophancy. We test this attribution across four model families and find it largely wrong: pretrained base models exhibit the same substitution pattern as their Instruct variants, averaging higher yield than Instruct. Using activation patching, we localize the corruption to a narrow mid-layer window where attention carries the causal weight and MLP contribution is negligible; patching above this window restores 96% of the clean-to-pressured P(correct) gap. The attack surface decomposes into two independent factors (channel framing and consensus strength) whose interaction produces a 47.5 percentage-point yield gap at majority consensus, preserved across jury sizes $N \in \{4, 5, 6\}$. Two converging activation-space interventions show that pressure suppresses clean-reasoning features rather than activating a new sycophancy circuit. A single correctly-arguing dissenter reduces yield by 54-73 percentage points across all framings tested, whereas the strongest prompt-level defense fails on attack variants outside its design surface. Mitigations should target the mechanism, structured dissent at the pipeline level, rather than prompt-level defenses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。