用失败模式指导数据筛选,让模型自己学会在哪跌倒就补哪。
Learning What to Fail On: Failure-Mode Contextual Bandits for Adversarial Data Curation

- 把对抗数据生成看作带反馈的智能选择问题,自动识别常见错误模式。
- 在多个数据集上提升模型准确率,最高达71.99%,优于已有方法。
- 无需人工标注,适合希望提升模型鲁棒性的研究者和工程师。
我们提出一种故障感知的对抗性检索增强框架,以提升自然语言理解的鲁棒性。不同于固定奖励阈值的选择方式,该方法将对抗性数据筛选建模为故障模式上下文强化学习问题。候选样本通过检索增强提示生成,经目标模型过滤,由大模型评判集合自动验证,并聚类为重复出现的故障模式。随机策略据此选择重训练的故障模式,并基于验证反馈的奖励更新,平衡鲁棒性提升、遗忘效应与数据成本。使数据策展者自身成为学习代理,实现训练轮次间对最有价值模型故障的自适应选择。在标准基准测试中,该方法将RoBERTa-base在SNLI上的准确率从88.48%提升至92.60%,在ANLI上从75.04%提升至80.95%,在MultiNLI上从54.67%提升至71.99%,持续优于先前对抗增强方法。进一步在FEVER事实验证任务上实现高达79.86%的FEVER得分和82.45%准确率(使用RoBERTa-large)。最后,我们提供了理论解释:在给定假设下,故障模式采样可降低捷径对齐梯度贡献,同时诱导有界分布漂移。结合检索、自动化验证、上下文带状故障选择与受控对抗重训练,本框架在无额外人工标注的前提下实现可扩展的鲁棒性提升。
原文摘要 · Abstract (English)
We introduce a failure-aware adversarial retrieval-augmented framework for improving robustness in natural language understanding. Rather than selecting synthetic examples with a fixed reward threshold, our method formulates adversarial data curation as a failure-mode contextual bandit problem. Candidate examples are generated with retrieval-augmented prompting, filtered by the current target model, automatically validated by an LLM judge ensemble, and clustered into recurring failure modes. A stochastic policy then selects which failure modes to sample for retraining, and is updated using validation-based reward that balances robustness gains, forgetting, and data cost. This makes the data curator itself the learning agent, enabling adaptive selection of the most useful model failures across training rounds. On standard benchmarks, our approach improves RoBERTa-base accuracy from 88.48% to 92.60% on SNLI, from 75.04% to 80.95% on ANLI, and from 54.67% to 71.99% on MultiNLI, while consistently outperforming prior adversarial augmentation methods. We further demonstrate transfer to FEVER fact verification, achieving up to 79.86\% FEVER score and 82.45\% accuracy with RoBERTa-large. Finally, we provide a theoretical interpretation showing that, under stated assumptions, failure-mode sampling can reduce shortcut-aligned gradient contributions while inducing bounded distributional drift. By combining retrieval, automated validation, contextual-bandit failure selection, and controlled adversarial retraining, our framework enables scalable robustness improvement without additional human annotation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。