研究多语言大模型如何有效遗忘敏感信息，发现遗忘可跨语言传递且可逆。
Multilingual Unlearning in LLMs: Transfer, Dynamics, and Reversibility
- 通过扩展基准测试支持五种语言，研究不同语言间遗忘效果的迁移规律。
- 在早期层保留共享语义空间，后期解码层实现知识抑制，非真正删除。
- 仅用一次推理时方向调整，即可恢复50%~90%被遗忘的知识，极具可逆性。
大语言模型可能记忆敏感信息，促使无需重新训练即可移除特定知识的遗忘方法发展。然而现有研究仍以英语为主。本文将TOFU基准拓展至五种语言，通过不同语言组合的微调、遗忘与查询实验，发现遗忘迁移能力差异显著：共享书写系统或语言家族的语言间迁移最强，且遗忘语言可预测哪些查询语言最易产生强迁移。层分析显示，遗忘主要作用于后期解码层，早期层的跨语言潜在空间基本保持不变。这表明遗忘并非真正删除知识，而是造成表面抑制。利用此结构，仅需一次推理时引导方向，即可在多种语言中恢复50%（Qwen）至90%（Gemma）的被遗忘知识。
原文摘要 · Abstract (English)
Large language models (LLMs) can memorize sensitive facts, motivating unlearning methods that remove targeted knowledge without costly retraining. However, unlearning research remains heavily English-centric. We study multilingual unlearning by extending the TOFU benchmark to five languages, and fine-tune, unlearn, and query our models with different permutations of languages. We find that unlearning transfer, the ability of an unlearned model to "forget" facts in languages other than the unlearning language, is highly variable: e.g., it is strongest between languages sharing scripts and families, and we show that the unlearning language predicts which query languages are most likely to yield the strongest transfer. Layer-wise analysis reveals that unlearning leaves the shared cross-lingual latent space largely intact in early layers, instead operating primarily in later decoding layers. This suggests that unlearning does not truly erase knowledge, but rather induces superficial suppression. Exploiting this structure, a single inference-time steering direction reverses much of this suppression across languages, recovering 50% (Qwen) and 90% (Gemma) of the unlearned knowledge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。