发现多语言模型拒绝有害请求的机制差异，定位到关键安全电路及其干预成本。
Decided Upstream, Written Late: Locating and Pricing the Cross-Lingual Refusal Circuit of a Multilingual MoE
- 通过追踪中间层特征方向，发现危害编码在不同语言间几乎一致（$L11$时余弦相似度≈0.9）
- 拒绝行为由后期生成阶段的特定专家写入电路控制，而非早期检测
- 干预该电路：削弱抑制器代价低且有效，强化写入器代价高昂，局部修改无效
多语言模型的安全对齐存在不平衡：一个在英语中可靠拒绝有害请求的模型，在低资源语言中却常顺从。我们在sarvam（一种印地语多语言专家混合模型）中机制性追溯这一差距，发现并非危害识别失败。危害在中层网络中以近乎语言不变的方向编码（$L11$处英-印余弦≈0.9），而上游操控该方向可因果控制拒绝行为。但检测方向与实际执行拒绝的改变正交，拒绝是后期生成过程中逐步组装而成，非单次前向传播读取。我们定位到一个可精确定位的、由专家写入器与注意力抑制器共同控制的电路，并量化其干预成本：减弱抑制器代价低且高效，增强写入器代价极高，对相关注意力头进行手术式修改无效。该电路结构及揭示方法在第二个无关的MoE模型中重复出现，但杠杆强度依赖具体架构。结果提供了一份成本测量的安全修复落点地图。
原文摘要 · Abstract (English)
Safety alignment in multilingual models is uneven: a model that reliably refuses a harmful request in English will often comply with the same request in a lower-resource language. We trace this gap mechanistically in sarvam, an Indic-multilingual mixture-of-experts reasoning model, and find it is not a failure to detect harm. Harm is encoded as an internal direction that is nearly language-invariant in mid-network (English-vs-Indic cosine ${\approx}0.9$ at $L11$), and steering that direction upstream causally controls refusal. But the detection direction is orthogonal to the change that actually writes the refusal, which is late and assembled over the course of generation rather than read off in a single forward pass. We attribute the write to a specific, localizable circuit, a mixture-of-experts writer held in check by an attention opposer and price every way of intervening on it: damping the opposer is cheap and effective, amplifying the writer is a cost wall, and surgical edits to the responsible heads do nothing. The circuit's organization, and the gradient method that exposes it, recur in a second, unrelated MoE model, while the lever's strength is architecture-specific. The result is a cost-measured map of where a multilingual safety repair can land, and what it costs
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。