通过模式识别检测大模型中的后门触发器，提升安全可信度。
Trojan Detection Through Pattern Recognition for Large Language Models
- 分三阶段检测：过滤、识别、验证，系统化定位后门
- 在TrojAI和RLHF数据集上实现高准确率检测，验证阶段关键
- 适合关注模型安全的开发者与研究人员使用
后门攻击可嵌入大语言模型的预训练、微调及上下文学习阶段，严重威胁模型对齐性。由于因果语言建模特性，触发器检测面临搜索空间巨大等挑战。本文提出多阶段检测框架：包括令牌过滤、触发器识别与验证。分析现有方法并提出两种基于输出logits的黑盒触发器逆向方法，分别采用束搜索与贪心解码。强调验证阶段的重要性，提出语义保持提示与特殊扰动以区分真实后门与相似对抗字符串。在TrojAI与RLHF中毒模型数据集上的评估显示效果良好。
原文摘要 · Abstract (English)
Trojan backdoors can be injected into large language models at various stages, including pretraining, fine-tuning, and in-context learning, posing a significant threat to the model's alignment. Due to the nature of causal language modeling, detecting these triggers is challenging given the vast search space. In this study, we propose a multistage framework for detecting Trojan triggers in large language models consisting of token filtration, trigger identification, and trigger verification. We discuss existing trigger identification methods and propose two variants of a black-box trigger inversion method that rely on output logits, utilizing beam search and greedy decoding respectively. We show that the verification stage is critical in the process and propose semantic-preserving prompts and special perturbations to differentiate between actual Trojan triggers and other adversarial strings that display similar characteristics. The evaluation of our approach on the TrojAI and RLHF poisoned model datasets demonstrates promising results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。