用预训练嵌入+传统模型提升大模型越狱检测准确率
Improved Large Language Model Jailbreak Detection via Pretrained Embeddings
- 结合检索专用嵌入与机器学习分类器检测越狱提示
- 在公开数据集上优于所有现有开源安全方案
- 适合需要强化LLM安全防护的系统开发者
大语言模型（LLMs）在客服机器人、编程助手及智能代理等场景中广泛应用，亟需安全防护。提示注入和越狱攻击旨在诱导模型生成违反安全、隐私或内容政策的内容。为防止模型产生有害回复或执行不当操作，模型所有者需在训练阶段设置防护，并集成额外工具阻止有害输出。越狱提示是促成此类行为的关键，因此识别越狱尝试至关重要。本文提出一种新方法：利用适用于检索的预训练文本嵌入，搭配传统机器学习分类算法，实现越狱提示检测。该方法在公开评测中超越所有现有开源安全工具。
原文摘要 · Abstract (English)
The adoption of large language models (LLMs) in many applications, from customer service chat bots and software development assistants to more capable agentic systems necessitates research into how to secure these systems. Attacks like prompt injection and jailbreaking attempt to elicit responses and actions from these models that are not compliant with the safety, privacy, or content policies of organizations using the model in their application. In order to counter abuse of LLMs for generating potentially harmful replies or taking undesirable actions, LLM owners must apply safeguards during training and integrate additional tools to block the LLM from generating text that abuses the model. Jailbreaking prompts play a vital role in convincing an LLM to generate potentially harmful content, making it important to identify jailbreaking attempts to block any further steps. In this work, we propose a novel approach to detect jailbreak prompts based on pairing text embeddings well-suited for retrieval with traditional machine learning classification algorithms. Our approach outperforms all publicly available methods from open source LLM security applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。