Sentinel模型可精准识别恶意提示注入，防护能力达当前顶尖水平。
Sentinel: SOTA model to protect against prompt injections
- 基于ModernBERT-large架构，通过多源数据精细微调
- 内部测试集准确率0.987，F1-score达0.980
- 适合需要高安全性的LLM应用部署者使用
大型语言模型（LLMs）虽日益强大，但仍易受提示注入攻击影响，此类攻击会诱导模型偏离原定指令。本文提出Sentinel，一个基于ModernBERT-large架构的新型检测模型（qualifire/prompt-injection-sentinel）。通过在涵盖开源与私有数据集的广泛且多样化的数据上进行微调，该模型整合了角色扮演、指令劫持及生成偏见内容等多种攻击类型，以及丰富良性指令样本，其中私有数据特别针对细微错误修正与真实世界误分类问题。在未见过的内部测试集上，Sentinel平均准确率达0.987，F1-score为0.980。同时，在公开基准测试中，其表现持续优于强基线模型如protectai/deberta-v3-base-prompt-injection-v2。本文详述了Sentinel的架构设计、数据集构建方法、训练策略及全面评估，凸显其卓越的检测性能。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly powerful but remain vulnerable to prompt injection attacks, where malicious inputs cause the model to deviate from its intended instructions. This paper introduces Sentinel, a novel detection model, qualifire/prompt-injection-sentinel, based on the \answerdotai/ModernBERT-large architecture. By leveraging ModernBERT's advanced features and fine-tuning on an extensive and diverse dataset comprising a few open-source and private collections, Sentinel achieves state-of-the-art performance. This dataset amalgamates varied attack types, from role-playing and instruction hijacking to attempts to generate biased content, alongside a broad spectrum of benign instructions, with private datasets specifically targeting nuanced error correction and real-world misclassifications. On a comprehensive, unseen internal test set, Sentinel demonstrates an average accuracy of 0.987 and an F1-score of 0.980. Furthermore, when evaluated on public benchmarks, it consistently outperforms strong baselines like protectai/deberta-v3-base-prompt-injection-v2. This work details Sentinel's architecture, its meticulous dataset curation, its training methodology, and a thorough evaluation, highlighting its superior detection capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。