提出新防御方法SCOUT,能识别隐蔽的后门攻击触发词。
SCOUT: A Defense Against Data Poisoning Attacks in Fine-Tuned Language Models
- 通过分析每个词移除对输出的影响,定位潜在后门触发词。
- 在多个数据集上成功检测出新型隐蔽攻击,保持正常输入准确率。
- 适合关注模型安全的医疗、金融等领域研究人员使用。
后门攻击通过嵌入隐藏触发词,在推理阶段操纵语言模型行为,对医疗等敏感领域构成重大威胁。现有防御方法虽能应对明显异常(如不相关触发词),但难以检测利用上下文合理性的隐蔽攻击。本文提出三种新型情境感知攻击:针对社交媒体成瘾分类的ViralApp攻击、诱导高血压诊断的Fever攻击、引导临床建议的Referral攻击,均利用领域知识与语义合理性,使触发词自然融入文本。为应对传统与新型攻击,我们提出 extbf{SCOUT(Saliency-based Classification Of Untrusted Tokens)},基于词级显著性分析识别后门触发词,通过测量单个词移除对目标标签输出逻辑值的影响构建显著性图谱。在SST-2、IMDB、AG News等基准数据集上,评估了针对BadNet、AddSent、SynBkd、StyleBkd及新攻击的性能,结果表明SCOUT有效检测复杂攻击,同时保持干净输入的准确率。
原文摘要 · Abstract (English)
Backdoor attacks create significant security threats to language models by embedding hidden triggers that manipulate model behavior during inference, presenting critical risks for AI systems deployed in healthcare and other sensitive domains. While existing defenses effectively counter obvious threats such as out-of-context trigger words and safety alignment violations, they fail against sophisticated attacks using contextually-appropriate triggers that blend seamlessly into natural language. This paper introduces three novel contextually-aware attack scenarios that exploit domain-specific knowledge and semantic plausibility: the ViralApp attack targeting social media addiction classification, the Fever attack manipulating medical diagnosis toward hypertension, and the Referral attack steering clinical recommendations. These attacks represent realistic threats where malicious actors exploit domain-specific vocabulary while maintaining semantic coherence, demonstrating how adversaries can weaponize contextual appropriateness to evade conventional detection methods. To counter both traditional and these sophisticated attacks, we present \textbf{SCOUT (Saliency-based Classification Of Untrusted Tokens)}, a novel defense framework that identifies backdoor triggers through token-level saliency analysis rather than traditional context-based detection methods. SCOUT constructs a saliency map by measuring how the removal of individual tokens affects the model's output logits for the target label, enabling detection of both conspicuous and subtle manipulation attempts. We evaluate SCOUT on established benchmark datasets (SST-2, IMDB, AG News) against conventional attacks (BadNet, AddSent, SynBkd, StyleBkd) and our novel attacks, demonstrating that SCOUT successfully detects these sophisticated threats while preserving accuracy on clean inputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。