用嵌入向量+机器学习检测恶意提示注入，效果优于现有方法
Embedding-based classifiers can detect prompt injection attacks
- 将提示词转为嵌入向量，用随机森林和XGBoost分类判断是否恶意
- 在多个数据集上准确率超开源神经网络模型，最高达98.7%
- 适合需要快速部署防护的LLM应用开发者使用
大型语言模型因强大的生成能力被广泛应用于各类组织。然而，它们易受各种对抗攻击，尤其是提示注入攻击，攻击者通过精心构造恶意提示诱导模型生成有害内容。本文提出一种基于嵌入的机器学习分类器新方法，以防御此类威胁。我们采用三种常用嵌入模型对恶意与正常提示生成嵌入表示，并使用机器学习分类器判断输入提示是否恶意。在多种传统机器学习方法中，基于随机森林和XGBoost的分类器表现最佳。实验结果表明，该方法在多个公开数据集上的检测性能超越现有开源的仅使用编码器的神经网络分类器，最高准确率达98.7%。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are seeing significant adoption in every type of organization due to their exceptional generative capabilities. However, LLMs are found to be vulnerable to various adversarial attacks, particularly prompt injection attacks, which trick them into producing harmful or inappropriate content. Adversaries execute such attacks by crafting malicious prompts to deceive the LLMs. In this paper, we propose a novel approach based on embedding-based Machine Learning (ML) classifiers to protect LLM-based applications against this severe threat. We leverage three commonly used embedding models to generate embeddings of malicious and benign prompts and utilize ML classifiers to predict whether an input prompt is malicious. Out of several traditional ML methods, we achieve the best performance with classifiers built using Random Forest and XGBoost. Our classifiers outperform state-of-the-art prompt injection classifiers available in open-source implementations, which use encoder-only neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。