用大模型自动生成对抗性对比样本，提升NLP模型鲁棒性。
Enhancing NLP Robustness and Generalization through LLM-Generated Contrast Sets: A Scalable Framework for Systematic Evaluation and Adversarial Training
- 用大模型自动构造多样对比集，替代人工设计。
- 在SNLI数据集上构建3000条对比样本，提升模型对扰动的适应能力。
- 适合关注模型鲁棒性与泛化能力的研究者和工程师。
标准NLP基准常无法捕捉由数据集特征和虚假关联引发的漏洞。对比集通过挑战模型在决策边界附近的判断来弥补这一缺陷，但传统方法依赖人工且多样性不足。本研究利用大语言模型自动化生成多样化对比集。基于SNLI数据集，构建了包含3,000个样本的对比集，用于评估和增强模型鲁棒性。在该对比集上微调后，模型在系统性扰动样本上的表现提升，标准测试准确率保持稳定，并对新类型扰动表现出适度的泛化改善。该自动化方法为NLP模型的系统性评估与对抗训练提供了可扩展方案，有助于应对实际应用中的泛化挑战，推动模型鲁棒性发展。
原文摘要 · Abstract (English)
Standard NLP benchmarks often fail to capture vulnerabilities stemming from dataset artifacts and spurious correlations. Contrast sets address this gap by challenging models near decision boundaries but are traditionally labor-intensive to create and limited in diversity. This study leverages large language models to automate the generation of diverse contrast sets. Using the SNLI dataset, we created a 3,000-example contrast set to evaluate and improve model robustness. Fine-tuning on these contrast sets enhanced performance on systematically perturbed examples, maintained standard test accuracy, and modestly improved generalization to novel perturbations. This automated approach offers a scalable solution for evaluating and improving NLP models, addressing systematic generalization challenges, and advancing robustness in real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。