用机器学习识别大模型越狱提示，能发现新攻击策略
Machine Learning for Detection and Analysis of Novel LLM Jailbreaks
- 微调BERT模型端到端检测越狱提示
- 在现有数据集上达到最佳准确率
- 发现提示结构中的自我指涉是越狱信号
大型语言模型（LLMs）存在多种漏洞，恶意用户可通过操纵输入文本诱导模型产生不当回应，这类攻击称为越狱提示。本文研究不同机器学习模型区分越狱提示与正常使用的能力，包括识别此前未见的攻击策略。结果表明，在现有数据集上，对双向编码器表示模型（BERT）进行端到端微调可取得最佳性能。通过可视化分析，发现提示结构中显式的自我指涉特征可能是越狱意图的指示信号。
原文摘要 · Abstract (English)
Large Language Models (LLMs) suffer from a range of vulnerabilities that allow malicious users to solicit undesirable responses through manipulation of the input text. These so-called jailbreak prompts are designed to trick the LLM into circumventing the safety guardrails put in place to keep responses acceptable to the developer's policies. In this study, we analyse the ability of different machine learning models to distinguish jailbreak prompts from genuine uses, including looking at our ability to identify jailbreaks that use previously unseen strategies. Our results indicate that using current datasets the best performance is achieved by fine tuning a Bidirectional Encoder Representations from Transformers (BERT) model end-to-end for identifying jailbreaks. We visualise the keywords that distinguish jailbreak from genuine prompts and conclude that explicit reflexivity in prompt structure could be a signal of jailbreak intention.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。