即使精确删除数据,仍可能被提取回敏感信息。
Unlearned but Not Forgotten: Data Extraction after Exact Unlearning in LLM
- 利用删除前后的模型输出差异,引导重构被删数据
- 在多个基准上提取成功率翻倍,医疗数据也有效
- 提醒需考虑删除前检查点的泄露风险
大型语言模型通常基于网络数据训练,可能包含有害或敏感个人信息。为应对隐私担忧,提出去学习方法以消除特定数据的影响。其中,精确去学习——从头重新训练剔除目标数据——被视为部署中缓解隐私风险的黄金标准。本文在预训练和去学习后均开放日志输出的实用场景下重新审视这一假设。针对此场景,我们提出一种新型数据提取攻击,利用删除前模型的信号指导删除后模型,揭示反映被删数据分布的模式。结合模型引导与词元过滤策略,攻击显著提升提取成功率,在MUSE、TOFU、WMDP等常见基准上实现性能翻倍。此外,我们在模拟医疗诊断数据集上验证了攻击有效性,凸显精确去学习在真实部署中可能反而加剧隐私泄露风险。基于此,我们呼吁评估去学习方法时应考虑更广泛的威胁模型,不仅关注去学习后模型,还需防范对先前检查点的恶意访问。代码已公开于:https://github.com/Nicholas0228/unlearned_data_extraction_llm。
原文摘要 · Abstract (English)
Large Language Models are typically trained on datasets collected from the web, which may inadvertently contain harmful or sensitive personal information. To address growing privacy concerns, unlearning methods have been proposed to remove the influence of specific data from trained models. Of these, exact unlearning -- which retrains the model from scratch without the target data -- is widely regarded the gold standard for mitigating privacy risks in deployment. In this paper, we revisit this assumption in a practical deployment setting where both the pre- and post-unlearning logits API are exposed, such as in open-weight scenarios. Targeting this setting, we introduce a novel data extraction attack that leverages signals from the pre-unlearning model to guide the post-unlearning model, uncovering patterns that reflect the removed data distribution. Combining model guidance with a token filtering strategy, our attack significantly improves extraction success rates -- doubling performance in some cases -- across common benchmarks such as MUSE, TOFU, and WMDP. Furthermore, we demonstrate our attack's effectiveness on a simulated medical diagnosis dataset to highlight real-world privacy risks associated with exact unlearning. In light of our findings, which suggest that unlearning may, in a contradictory way, increase the risk of privacy leakage during real-world deployments, we advocate for evaluation of unlearning methods to consider broader threat models that account not only for post-unlearning models but also for adversarial access to prior checkpoints. Code is publicly available at: https://github.com/Nicholas0228/unlearned_data_extraction_llm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。