arXiv:2608.18089cs.CLcs.AI2026-08

不重新训练,让AI在非洲小语种中学会拒绝有害请求。

Latent Space Refusal Anchoring for Low-Resource African Languages: Mechanistic Safety Recovery Without Retraining

  • 从英文提示中提取拒绝指令方向,直接注入推理时的模型残差流
  • 在四种模型上恢复安全,误拒率低于0.08,大模型性能损失极小
  • 适用于低资源非洲语言,但阿拉伯语因语义空间不匹配失效

指令微调模型在英语中会拒绝有害请求,但在约鲁巴语、伊博语、伊加拉语和豪萨语中却会顺从。这表明拒绝机制存在于残差流中,但对低资源语言无法激活。传统恢复方法需目标语言标注数据和重训练,难以规模化应用于多数非洲语言。本文提出无需训练的潜空间拒绝锚定(LSR-Anchoring),从英语提示中提取拒绝方向,并在推理时将其固定到残差流。主要变体均值激活引导(MAS)在四种架构(Llama-3-8B、Llama-3.1-70B、Mistral-7B-Instruct、Qwen2.5-7B)上有效,其中在Mistral和Qwen上实现安全恢复且良性退化低于0.08;在Llama-3-8B上过度纠正,合法请求退化(DPL)达1.00。为此引入基于稀疏自编码器(SAE)的引导(SDS),用单一SAE特征替代密集方向,使KL散度降低3.5–7倍,无良性崩溃。四门非洲语言正向迁移,但阿拉伯语在所有架构与强度下均失败,表明存在几何不匹配而非基线问题。大规模多任务语言理解(MMLU)准确率下降始终低于0.35个百分点。

原文摘要 · Abstract (English)

Instruction-tuned models often refuse harmful requests in English but comply with the same requests in Yoruba, Igbo, Igala, and Hausa. This suggests that the refusal mechanism is present in the residual stream but fails to activate for low-resource inputs. Recovering it normally requires labelled target-language data and retraining, neither of which is available at scale for most African languages. We introduce Latent Space Refusal Anchoring (LSR-Anchoring), a training-free method that extracts the refusal direction from English prompts and clamps it onto the residual stream at inference time. The primary variant, Mean-Activation Steering (MAS), operates across the four architectures we tested: Llama-3-8B, Llama-3.1-70B, Mistral-7B-Instruct, and Qwen2.5-7B. On Mistral and Qwen it recovers safety with benign degradation below 0.08. On Llama-3-8B it overcorrects, with Degraded Performance on Legitimate prompts (DPL) reaching 1.00. We address this with SAE-Derived Steering (SDS), which replaces the dense mean-difference direction with a single Sparse Autoencoder (SAE) feature and reduces Kullback-Leibler (KL) divergence by 3.5-7x without benign collapse. Four languages transfer positively, but Arabic fails on every architecture and at every steering magnitude, indicating a geometric mismatch rather than a baseline effect. Massive Multitask Language Understanding (MMLU) accuracy drops remain below 0.35 percentage points at every effective steering magnitude.

模型安全低资源语言推理优化AI伦理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。