提出FRS方法,高效认证语言模型抗后门攻击能力。
Certifying Language Model Robustness with Fuzzed Randomized Smoothing: An Efficient Defense Against Backdoor Attacks
- 结合模糊测试与双阶段参数平滑,主动识别文本漏洞
- 无需原始污染数据即可实现更广的认证鲁棒半径
- 适用于高可靠性场景下的模型安全验证
预训练语言模型(PLMs)的广泛应用使其面临文本后门攻击的威胁,尤其是预训练阶段植入的攻击。这类攻击在多个下游任务中隐秘生效,危害重大。尽管认证鲁棒性至关重要,但现有防御方法受限于文本数据的高维性和强依赖性,且缺乏对原始污染训练数据的访问。为此,我们提出新颖的模糊随机平滑(Fuzzed Randomized Smoothing, FRS)方法,用于高效认证语言模型对抗后门攻击的能力。FRS融合软件鲁棒性认证技术与双阶段模型参数平滑,利用蒙特卡洛树搜索在达马胡-列文斯坦空间中主动模糊探测脆弱文本片段,实现目标化、高效的文本随机化,且在模型平滑过程中无需访问污染数据。理论分析表明,FRS相较于现有方法可获得更广的认证鲁棒半径。在多种数据集、模型配置和攻击策略下的大量实验验证了FRS在防御效率、准确率和鲁棒性方面的优越性。
原文摘要 · Abstract (English)
The widespread deployment of pre-trained language models (PLMs) has exposed them to textual backdoor attacks, particularly those planted during the pre-training stage. These attacks pose significant risks to high-reliability applications, as they can stealthily affect multiple downstream tasks. While certifying robustness against such threats is crucial, existing defenses struggle with the high-dimensional, interdependent nature of textual data and the lack of access to original poisoned pre-training data. To address these challenges, we introduce \textbf{F}uzzed \textbf{R}andomized \textbf{S}moothing (\textbf{FRS}), a novel approach for efficiently certifying language model robustness against backdoor attacks. FRS integrates software robustness certification techniques with biphased model parameter smoothing, employing Monte Carlo tree search for proactive fuzzing to identify vulnerable textual segments within the Damerau-Levenshtein space. This allows for targeted and efficient text randomization, while eliminating the need for access to poisoned training data during model smoothing. Our theoretical analysis demonstrates that FRS achieves a broader certified robustness radius compared to existing methods. Extensive experiments across various datasets, model configurations, and attack strategies validate FRS's superiority in terms of defense efficiency, accuracy, and robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。