微调数据特性可能意外引入模型漏洞，影响安全防护。
Accidental Vulnerability: Factors in Fine-Tuning that Shift Model Safeguards
- 分析微调数据的语言特征、语义相似度和毒性等因子
- 发现数据特性与攻击成功率显著相关，导致角色偏移
- 强调数据设计对模型对齐的关键作用，适合安全研究者
随着大语言模型广泛应用，其对抗攻击脆弱性成为主要关切。尽管在领域特定数据集上微调可提升性能，却可能意外引入模型漏洞。本文研究意外漏洞——由微调数据特性引发的未预期脆弱性。通过多组实验，识别语言特征、语义相似度与毒性等潜在关联因素。评估微调后模型的对抗鲁棒性，分析角色转变与可解释性特征，揭示数据因素如何影响攻击成功率。进一步探究因果关系，为对抗防御策略提供新见解，强调数据设计在维持模型对齐中的关键作用。代码已公开于 https://github.com/psyonp/accidental_vulnerability。
原文摘要 · Abstract (English)
As large language models (LLMs) gain popularity, their vulnerability to adversarial attacks emerges as a primary concern. While fine-tuning models on domain-specific datasets is often employed to improve model performance, it can inadvertently introduce vulnerabilities within the underlying model. In this work, we investigate Accidental Vulnerability, unexpected vulnerabilities arising from characteristics of fine-tuning data. We begin by identifying potential correlation factors such as linguistic features, semantic similarity, and toxicity across multiple experimental datasets. We then evaluate the adversarial robustness of these fine-tuned models, analyzing persona shifts and interpretability traits to understand how dataset factors contribute to attack success rates. Lastly, we explore causal relationships that offer new insights into adversarial defense strategies, highlighting the crucial role of dataset design in preserving model alignment. Our code is available at https://github.com/psyonp/accidental_vulnerability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。