针对自然语言推理中的数据偏差，提出多头去偏与对比学习方法提升模型泛化能力。
Multi-head attention debiasing and contrastive learning for mitigating Dataset Artifacts in Natural Language Inference
- 设计多头去偏架构，结合对比学习消除文本长度、词汇重叠等四类偏差
- 在SNLI验证集上，整体错误率从14.19%降至10.42%，各类偏差均显著改善
- 特别提升中性关系判别能力，适合关注模型公平性与鲁棒性的研究者
尽管自然语言推理（NLI）模型在基准数据集上表现优异，但其是否真正理解任务仍存疑，可能过度依赖数据集中的特定模式。通过对斯坦福自然语言推理（SNLI）数据集的深入分析，我们发现四类主要数据偏差：长度相关模式、词汇重叠、子集关系和否定模式。通过细粒度分析9,782个验证样本，提出新型结构化去偏方法。该多头去偏架构在所有偏差类别中均有显著提升：长度偏差准确率从86.03%升至90.06%，词汇重叠从91.88%增至93.13%，子集偏差从95.43%升至96.49%，否定偏差从88.69%增至94.64%。总体错误率由14.19%降至10.42%，同时保持对无偏样本的高性能。对1,026个错误案例的分析显示，中性关系判别能力明显增强，这是传统NLI系统最薄弱环节之一。
原文摘要 · Abstract (English)
While Natural Language Inference (NLI) models have achieved high performances on benchmark datasets, there are still concerns whether they truly capture the intended task, or largely exploit dataset artifacts. Through detailed analysis of the Stanford Natural Language Inference (SNLI) dataset, we have uncovered complex patterns of various types of artifacts and their interactions, leading to the development of our novel structural debiasing approach. Our fine-grained analysis of 9,782 validation examples reveals four major categories of artifacts: length-based patterns, lexical overlap, subset relationships, and negation patterns. Our multi-head debiasing architecture achieves substantial improvements across all bias categories: length bias accuracy improved from 86.03% to 90.06%, overlap bias from 91.88% to 93.13%, subset bias from 95.43% to 96.49%, and negation bias from 88.69% to 94.64%. Overall, our approach reduces the error rate from 14.19% to 10.42% while maintaining high performance on unbiased examples. Analysis of 1,026 error cases shows significant improvement in handling neutral relationships, traditionally one of the most challenging areas for NLI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。