arXiv:2410.21337cs.CLcs.AI2024-10被引 20

微调大模型可精准识别提示注入攻击,准确率达99.13%。

Fine-tuned Large Language Models (LLMs): Improved Prompt Injection Attacks Detection

  • 用标注数据微调XLM-RoBERTa模型提升检测能力
  • 实现99.13%准确率、100%精确率的检测效果
  • 适合关注LLM安全与防御的研究者参考

大型语言模型(LLMs)在处理各类语言任务方面能力显著提升,但其应用极易受到提示注入攻击的威胁。此类攻击通过精心设计的输入提示,使模型偏离原始指令,从而执行未授权操作,可能导致数据泄露、输出偏见或有害响应。本文研究了提示注入攻击的安全漏洞,采用两种方法进行检测:预训练模型与微调模型。首先使用未微调的XLM-RoBERTa模型进行零样本分类;随后,利用Hugging Face上deepset提供的特定任务标注数据集对模型进行监督微调。经严格实验评估,微调后模型在测试集上达到99.13%准确率、100%精确率、98.33%召回率和99.15%F1分数,表明该方法在检测提示注入攻击方面具有高度有效性。

原文摘要 · Abstract (English)

Large language models (LLMs) are becoming a popular tool as they have significantly advanced in their capability to tackle a wide range of language-based tasks. However, LLMs applications are highly vulnerable to prompt injection attacks, which poses a critical problem. These attacks target LLMs applications through using carefully designed input prompts to divert the model from adhering to original instruction, thereby it could execute unintended actions. These manipulations pose serious security threats which potentially results in data leaks, biased outputs, or harmful responses. This project explores the security vulnerabilities in relation to prompt injection attacks. To detect whether a prompt is vulnerable or not, we follows two approaches: 1) a pre-trained LLM, and 2) a fine-tuned LLM. Then, we conduct a thorough analysis and comparison of the classification performance. Firstly, we use pre-trained XLM-RoBERTa model to detect prompt injections using test dataset without any fine-tuning and evaluate it by zero-shot classification. Then, this proposed work will apply supervised fine-tuning to this pre-trained LLM using a task-specific labeled dataset from deepset in huggingface, and this fine-tuned model achieves impressive results with 99.13\% accuracy, 100\% precision, 98.33\% recall and 99.15\% F1-score thorough rigorous experimentation and evaluation. We observe that our approach is highly efficient in detecting prompt injection attacks.

提示注入大模型安全微调检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。