arXiv:2601.02700cs.CLcs.AI2026-01

针对问答模型在对抗样本下的脆弱性,提出系统性分析与针对性改进方案。

Adversarial Question Answering Robustness: A Multi-Level Error Analysis and Mitigation Study

  • 通过多层级错误分类,发现否定混淆和实体替换是主要失败模式。
  • 小模型存在容量瓶颈,大模型可同时提升干净数据与对抗数据表现。
  • 基于命名实体的对比学习方法显著缩小对抗差距,性能接近正常水平。

问答系统在SQuAD等标准基准上表现优异,但对对抗样本仍显脆弱。本研究基于AddSent对抗数据集,系统评估了不同规模Transformer模型的鲁棒性,并探索针对性缓解策略。通过五种互补的错误分类框架进行多层次分析,识别出否定混淆和实体替换为主要失效模式。系统评估对抗微调比例后发现,80%清洁数据 + 20%对抗数据为最优配置。数据增强实验揭示小模型存在容量瓶颈。从ELECTRA-small(14M参数)扩展到ELECTRA-base(110M参数)后,彻底消除鲁棒性与准确率的权衡,在清洁与对抗数据上均实现显著提升。实施三种针对性缓解策略,其中实体感知对比学习表现最佳:AddSent精确匹配达89.89%,SQuAD精确匹配达90.73%,对抗差距缩小至94.9%。据我们所知,这是首个结合全面语言学错误分析与命名实体识别(NER)引导的对比学习来应对对抗问答的研究,证明针对性缓解可实现清洁与对抗性能近乎一致。

原文摘要 · Abstract (English)

Question answering (QA) systems achieve impressive performance on standard benchmarks like SQuAD, but remain vulnerable to adversarial examples. This project investigates the adversarial robustness of transformer models on the AddSent adversarial dataset through systematic experimentation across model scales and targeted mitigation strategies. We perform comprehensive multi-level error analysis using five complementary categorization schemes, identifying negation confusion and entity substitution as the primary failure modes. Through systematic evaluation of adversarial fine-tuning ratios, we identify 80% clean + 20% adversarial data as optimal. Data augmentation experiments reveal a capacity bottleneck in small models. Scaling from ELECTRA-small (14M parameters) to ELECTRA-base (110M parameters) eliminates the robustness-accuracy trade-off, achieving substantial improvements on both clean and adversarial data. We implement three targeted mitigation strategies, with Entity-Aware contrastive learning achieving best performance: 89.89% AddSent Exact Match (EM) and 90.73% SQuAD EM, representing 94.9% closure of the adversarial gap. To our knowledge, this is the first work integrating comprehensive linguistic error analysis with Named Entity Recognition (NER)-guided contrastive learning for adversarial QA, demonstrating that targeted mitigation can achieve near-parity between clean and adversarial performance.

问答系统对抗鲁棒性对比学习命名实体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。