攻击者可利用微调接口的损失信号，高效生成欺骗大模型的恶意提示。
Fun-tuning: Characterizing the Vulnerability of Proprietary LLMs to Optimization-based Prompt Injection Attacks via the Fine-Tuning Interface
- 通过微调接口返回的损失信息，用贪心算法优化生成对抗性提示。
- 在Gemini模型上攻击成功率达65%至82%，验证了威胁可行性。
- 适合关注模型安全与接口设计的研究者及开发者参考。
我们揭示了一种针对闭源大语言模型的新威胁：攻击者可利用远程微调接口返回的类似损失值信息，指导对抗性提示的优化搜索。该微调接口由厂商提供，虽为开发者带来便利，却也泄露足够信号供攻击者构造恶意提示。通过对Gemini微调API的实验分析，我们发现其返回的损失值能有效支持离散优化，结合PurpleLlama基准测试，在Google Gemini系列模型上实现65%至82%的攻击成功率。该攻击凸显了功能实用性与安全性之间的经典权衡——微调接口既提升效率，也引入强大攻击面。
原文摘要 · Abstract (English)
We surface a new threat to closed-weight Large Language Models (LLMs) that enables an attacker to compute optimization-based prompt injections. Specifically, we characterize how an attacker can leverage the loss-like information returned from the remote fine-tuning interface to guide the search for adversarial prompts. The fine-tuning interface is hosted by an LLM vendor and allows developers to fine-tune LLMs for their tasks, thus providing utility, but also exposes enough information for an attacker to compute adversarial prompts. Through an experimental analysis, we characterize the loss-like values returned by the Gemini fine-tuning API and demonstrate that they provide a useful signal for discrete optimization of adversarial prompts using a greedy search algorithm. Using the PurpleLlama prompt injection benchmark, we demonstrate attack success rates between 65% and 82% on Google's Gemini family of LLMs. These attacks exploit the classic utility-security tradeoff - the fine-tuning interface provides a useful feature for developers but also exposes the LLMs to powerful attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。