用单位矩阵权重和多间隔损失提升BERT抗攻击能力
Unitary Multi-Margin BERT for Robust Natural Language Processing
- 将单位权值与多间隔损失结合，增强模型鲁棒性
- 攻击后准确率提升5.3%，达73.8%，且攻击前表现不降
- 仅调一个参数即可平衡防御强度与原始性能，适合实际部署
深度学习面临的对抗攻击威胁使众多关键NLP系统面临风险。为解决计算效率低的防御方法问题，本文提出一种通用新方法：将单位权值与多间隔损失结合，显著提升双向编码器表示模型（BERT）的鲁棒性。该方法在保持攻击前准确率竞争力的同时，使攻击后分类准确率提升5.3个百分点，达到73.8%。此外，通过单一标量参数可灵活调节攻击前后准确率的权衡，以适配不同应用场景的需求。
原文摘要 · Abstract (English)
Recent developments in adversarial attacks on deep learning leave many mission-critical natural language processing (NLP) systems at risk of exploitation. To address the lack of computationally efficient adversarial defense methods, this paper reports a novel, universal technique that drastically improves the robustness of Bidirectional Encoder Representations from Transformers (BERT) by combining the unitary weights with the multi-margin loss. We discover that the marriage of these two simple ideas amplifies the protection against malicious interference. Our model, the unitary multi-margin BERT (UniBERT), boosts post-attack classification accuracies significantly by 5.3% to 73.8% while maintaining competitive pre-attack accuracies. Furthermore, the pre-attack and post-attack accuracy tradeoff can be adjusted via a single scalar parameter to best fit the design requirements for the target applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。