通过动态掩码增强模型抗文本攻击能力,提升鲁棒性。
Defensive Dual Masking for Robust Adversarial Defense
- 训练时插入[MASK]增强模型抗干扰能力
- 推理时动态替换可疑词为[MASK],准确率显著提升
- 适用于大模型,适合高安全场景应用
近年来,自然语言处理模型对文本对抗攻击日益敏感,此类攻击通过细微输入扰动误导模型。本文提出防御性双重掩码(Defensive Dual Masking, DDM)算法,通过在训练中策略性插入[MASK]标记,使模型更好地应对对抗扰动。推理时,将潜在攻击性词汇动态替换为[MASK],在保留语义核心的同时消除威胁。理论分析表明,选择性掩码机制提升了模型识别与抑制对抗操纵的能力。在多个基准数据集和攻击方法上的实证评估显示,DDM consistently 超越现有最先进防御方法,显著提升模型准确率与鲁棒性。该方法同样有效应用于大型语言模型(LLMs),为大规模NLP应用提供可扩展的防御方案。
原文摘要 · Abstract (English)
The field of textual adversarial defenses has gained considerable attention in recent years due to the increasing vulnerability of natural language processing (NLP) models to adversarial attacks, which exploit subtle perturbations in input text to deceive models. This paper introduces the Defensive Dual Masking (DDM) algorithm, a novel approach designed to enhance model robustness against such attacks. DDM utilizes a unique adversarial training strategy where [MASK] tokens are strategically inserted into training samples to prepare the model to handle adversarial perturbations more effectively. During inference, potentially adversarial tokens are dynamically replaced with [MASK] tokens to neutralize potential threats while preserving the core semantics of the input. The theoretical foundation of our approach is explored, demonstrating how the selective masking mechanism strengthens the model's ability to identify and mitigate adversarial manipulations. Our empirical evaluation across a diverse set of benchmark datasets and attack mechanisms consistently shows that DDM outperforms state-of-the-art defense techniques, improving model accuracy and robustness. Moreover, when applied to Large Language Models (LLMs), DDM also enhances their resilience to adversarial attacks, providing a scalable defense mechanism for large-scale NLP applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。