用密码数据微调大模型会泄露密码，可被精准找回并删除。
Leaking LoRa: An Evaluation of Password Leaks and Knowledge Storage in Large Language Models
- 用LoRA微调模型时注入密码，37个测试密码可被成功恢复。
- 密码信息集中在少数模型层，定位明确。
- 用罗马编辑法可彻底清除密码痕迹，恢复为0泄露。
为在特定应用中高效部署大语言模型（LLMs），常采用微调技术提升专业任务表现。此过程可能涉及用户敏感数据的微调，包括密码。尽管不推荐，但用户向客服发送密码的情况仍存在，可能导致密码泄露。本研究使用低秩适配（LoRA）对大模型进行微调，训练数据包含客户支持记录及来自RockYou密码词表的前200个密码。实验发现其中37个密码被成功恢复。通过因果追踪分析，发现密码信息主要存在于少数模型层。进一步采用秩一模型编辑（ROME）方法移除密码信息，使密码恢复数量从37降至0，证明了信息可被有效擦除。
原文摘要 · Abstract (English)
To effectively deploy Large Language Models (LLMs) in application-specific settings, fine-tuning techniques are applied to enhance performance on specialized tasks. This process often involves fine-tuning on user data data, which may contain sensitive information. Although not recommended, it is not uncommon for users to send passwords in messages, and fine-tuning models on this could result in passwords being leaked. In this study, a Large Language Model is fine-tuned with customer support data and passwords from the RockYou password wordlist using Low-Rank Adaptation (LoRA). Out of the first 200 passwords from the list, 37 were successfully recovered. Further, causal tracing is used to identify that password information is largely located in a few layers. Lastly, Rank One Model Editing (ROME) is used to remove the password information from the model, resulting in the number of passwords recovered going from 37 to 0.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。