轻量级可解释安全防护机制，能准确识别不安全提示并给出理由。
A Lightweight Explainable Guardrail for Prompt Safety
- 多任务学习联合训练分类与解释模型，提升可解释性。
- 在三个数据集上表现优于或等同于当前最优方法，模型更小。
- 适合需要安全可控、透明决策的AI应用开发者使用。
我们提出一种轻量级可解释安全防护机制（LEG），用于检测不安全提示。LEG采用多任务学习架构，同时学习提示分类和解释分类，后者标注出解释整体判断结果的安全/不安全关键词。该方法在合成解释数据上训练，这些数据通过一种新策略生成，以抵消大模型的确认偏见。训练过程引入一种新损失函数，利用全局解释信号作为弱监督，结合交叉熵、焦点损失与基于不确定性的加权策略。尽管模型规模远小于现有方法，LEG在三个数据集上的提示分类与可解释性表现均达到或超过当前最优水平，且在域内与域外均有效。
原文摘要 · Abstract (English)
We propose a lightweight explainable guardrail (LEG) method to detect unsafe prompts. LEG uses a multi-task learning architecture to jointly learn a prompt classifier and an explanation classifier, where the latter labels prompt words that explain the safe/unsafe overall decision. LEG is trained on synthetic explanation data, which is generated using a novel strategy that counteracts the confirmation biases of LLMs. Lastly, LEG's training process uses a novel loss that captures global explanation signals as a weak supervision and combines cross-entropy and focal losses with uncertainty-based weighting. LEG obtains equivalent or better performance than the state-of-the-art for both prompt classification and explainability, both in-domain and out-of-domain on three datasets, despite the fact that its model size is considerably smaller than current approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。