arXiv:2606.13439cs.CLcs.LG2026-06中稿 · NETYS 2026 - 14th …

提出二阶鲁棒性边界方法,提升模型对抗词替换攻击的可信防御能力。

S-GBT: Smooth Growth Bound Tensor for Certified Robustness Against Word Substitution Attacks in NLP

论文配图:S-GBT: Smooth Growth Bound Tensor for Certified Robustness Against Word Substitution Attacks in NLP
图 1 · 摘自论文原文
  • 通过控制梯度变化率(曲率)构建更紧的鲁棒性边界
  • 在多个数据集上使认证鲁棒准确率提升最高达23.4%
  • 适用于LSTM与CNN,训练时直接融入正则化项

尽管自然语言处理取得进展,模型仍易受词替换攻击。现有防御多关注一阶敏感性,忽略梯度随输入变化的演化特性——即曲率。当梯度剧烈变化时,模型仍可能失效。本文提出平滑增长边界张量(S-GBT),一种基于二阶信息的防御方法,对海森矩阵元素逐项施加边界约束,并给出严格的理论证明。训练中引入正则化项以最小化这些边界,使输出变化由线性项与二次项共同限制。S-GBT适用于长短期记忆网络(LSTM)和卷积神经网络(CNN),可直接集成于训练目标。在多个基准数据集上的实验表明,结合一阶与二阶正则化后,认证鲁棒准确率相较先前方法最高提升23.4%,且干净准确率保持竞争力。结果表明,同时控制梯度及其变化是提升模型鲁棒性的有效方向。

原文摘要 · Abstract (English)

Despite recent progress in Natural Language Processing (NLP), models remain vulnerable to word substitution attacks. Most existing defenses focus on first order sensitivity and measure how much the output changes when the input is slightly perturbed. However, they ignore how this sensitivity evolves, which is described by curvature. When gradients vary sharply, models can still fail. This paper introduces the Smooth Growth Bound Tensor (S-GBT), a second order method that bounds the Hessian element-wise, for which we provide formal theoretical proofs on the resulting robustness bounds. A regularization term is added during training to minimize these bounds. This yields tighter certified robustness against word substitution attacks. The change in the output under word substitution is bounded by both a linear term and a quadratic term. S-GBT is derived for two architectures: Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNN). The method is integrated directly into the training objective. Its effectiveness is evaluated on multiple benchmark datasets. The results show that combining first and second order regularization improves certified robust accuracy by up to 23.4% compared to prior methods, while clean accuracy remains competitive. These findings indicate that controlling both the gradient and its variation is a promising direction for building more robust models.

NLP安全词替换攻击二阶鲁棒性认证防御

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。