提出跨语言信息遗忘评估新方法，解决多语言大模型隐私泄露难题
Knowledge Beyond Language: Bridging the Gap in Multilingual Machine Unlearning Evaluation
- 设计双指标体系评估跨语言信息遗忘效果
- 发现多语言遗忘存在独特信息扩散现象
- 适合关注多语言模型隐私安全的研究者
尽管大型语言模型在商业服务中日益普及，但其存在敏感个人身份信息（PII）泄露等隐私风险。针对基于多语言语料训练的大型语言模型，多语言机器遗忘（MMU）旨在消除多种语言中的信息。然而，以往的MMU评估未能捕捉信息在不同语言间的分布特征，主要局限于单语言评估协议的直接扩展。为此，我们提出两种评估跨语言信息传播的新指标：知识可分离度评分（KSS）和知识持续度评分（KPS）。KSS衡量多语言环境下整体遗忘质量，而KPS则更具体地评估不同语言对间信息移除的一致性。我们利用这些指标对多种遗忘方法在多语言设置下进行了评估，并开展了全面分析。研究揭示了仅在MMU中出现的独特现象，为多语言遗忘评估提供了新视角。
原文摘要 · Abstract (English)
While LLMs are increasingly used in commercial services, they pose privacy risks such as leakage of sensitive personally identifiable information (PII). For LLMs trained on multilingual corpora, Multilingual Machine Unlearning (MMU) aims to remove information across multiple languages. However, prior MMU evaluations fail to capture such cross-linguistic distribution of information, being largely limited to direct extensions of per-language evaluation protocols. To this end, we propose two metrics to evaluate the information spread across languages: the Knowledge Separability Score (KSS) and the Knowledge Persistence Score (KPS). KSS measures the overall unlearning quality across multiple languages, while KPS more specifically aims to assess consistent removal of information among different language pairs. We evaluated various unlearning methods in the multilingual setting with these metrics and conducted comprehensive analyses. Through our investigation, we provide insights into unique phenomena exclusive to MMU and offer a new perspective on MMU evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。