用开源大模型微调时，你的数据可能被悄悄偷走
Be Careful When Fine-tuning On Open-Source LLMs: Your Fine-tuning Data Could Be Secretly Stolen!
- 通过后门训练，可从微调模型中提取私有数据
- 实测可恢复76.3%的微调数据，理想下达94.9%
- 适合关注数据安全的开发者与研究者
使用开源大模型微调私有数据已成为常见做法。然而我们发现一种新风险：开源模型作者可在仅拥有黑盒访问权限的情况下，通过简单后门训练，从下游微调模型中提取原始私有数据。我们在4个主流开源模型（参数量3B至32B）和2个下游数据集上进行实验，结果显示在实际场景中，5,000个样本中的76.3%微调数据（查询）可被完全恢复；在更理想条件下，成功率可达94.9%。我们尝试了基于检测的防御策略，但发现其可被改进攻击绕过。该结果揭示了微调中的新型数据泄露风险，亟需后续研究应对。代码与数据已公开于https://github.com/thu-coai/Backdoor-Data-Extraction。
原文摘要 · Abstract (English)
Fine-tuning on open-source Large Language Models (LLMs) with proprietary data is now a standard practice for downstream developers to obtain task-specific LLMs. Surprisingly, we reveal a new and concerning risk along with the practice: the creator of the open-source LLMs can later extract the private downstream fine-tuning data through simple backdoor training, only requiring black-box access to the fine-tuned downstream model. Our comprehensive experiments, across 4 popularly used open-source models with 3B to 32B parameters and 2 downstream datasets, suggest that the extraction performance can be strikingly high: in practical settings, as much as 76.3% downstream fine-tuning data (queries) out of a total 5,000 samples can be perfectly extracted, and the success rate can increase to 94.9% in more ideal settings. We also explore a detection-based defense strategy but find it can be bypassed with improved attack. Overall, we highlight the emergency of this newly identified data breaching risk in fine-tuning, and we hope that more follow-up research could push the progress of addressing this concerning risk. The code and data used in our experiments are released at https://github.com/thu-coai/Backdoor-Data-Extraction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。