大模型删数据后仍留痕迹,可被轻易检测到。
Unlearning Isn't Invisible: Detecting Unlearning Traces in LLMs from Model Outputs
- 通过分析模型输出和中间激活,发现删除数据后仍有可识别的残留特征。
- 即使使用无关问题提问,也能以90%以上准确率检测出模型是否被删过数据。
- 越大模型越容易被检测,揭示了删除隐私信息后的潜在泄露风险。
大型语言模型(LLMs)的机器遗忘(MU)旨在移除特定不良数据或知识,同时保持标准任务性能。尽管遗忘对保护数据隐私、维护版权及减轻社会技术危害至关重要,我们发现遗忘后存在新漏洞:遗忘痕迹可被检测。研究发现,遗忘会在模型行为和内部表征中留下持久的“指纹”,这些痕迹可从输出响应中识别,甚至在未涉及遗忘内容的提示下亦可探测。具体而言,仅需使用预测逻辑值或文本输出,简单监督分类器即可判断模型是否经过遗忘处理。进一步分析表明,这些痕迹嵌入于中间激活中,并非线性传播至最终层,在激活空间中形成低维可学习流形。大量实验显示,即便面对与遗忘无关的输入,遗忘痕迹的检测准确率仍超90%,且更大的模型表现出更强的可检测性。结果表明,遗忘会留下可测量的签名,当模型被确认为已遗忘时,可能面临反向推断被遗忘信息的风险。
原文摘要 · Abstract (English)
Machine unlearning (MU) for large language models (LLMs), commonly referred to as LLM unlearning, seeks to remove specific undesirable data or knowledge from a trained model, while maintaining its performance on standard tasks. While unlearning plays a vital role in protecting data privacy, enforcing copyright, and mitigating sociotechnical harms in LLMs, we identify a new vulnerability post-unlearning: unlearning trace detection. We discover that unlearning leaves behind persistent "fingerprints" in LLMs, detectable traces in both model behavior and internal representations. These traces can be identified from output responses, even when prompted with forget-irrelevant inputs. Specifically, even a simple supervised classifier can determine whether a model has undergone unlearning, using only its prediction logits or even its textual outputs. Further analysis shows that these traces are embedded in intermediate activations and propagate nonlinearly to the final layer, forming low-dimensional, learnable manifolds in activation space. Through extensive experiments, we demonstrate that unlearning traces can be detected with over 90% accuracy even under forget-irrelevant inputs, and that larger LLMs exhibit stronger detectability. These findings reveal that unlearning leaves measurable signatures, introducing a new risk of reverse-engineering forgotten information when a model is identified as unlearned, given an input query.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。