用AI自动生成对抗样本，让自然语言推理模型更抗干扰。
VAULT: Vigilant Adversarial Updates via LLM-Driven Retrieval-Augmented Generation for NLI
- 结合语义与词法检索，用大模型生成高逼真对抗假设。
- 在多个数据集上将RoBERTa模型准确率提升超17个百分点。
- 全流程自动化，适合快速提升NLI模型鲁棒性的人工智能研究者。
我们提出VAULT，一个全自动的对抗性RAG流水线，通过三阶段系统性发现并修复自然语言推理（NLI）模型的弱点：检索、对抗生成和迭代重训练。首先，采用平衡的少样本检索，利用BGE语义相似度和BM25词法相似度对前提进行匹配。其次，将检索到的上下文整合为大模型提示，生成对抗性假设，并由大模型集成进行标签一致性验证。最后，将验证后的对抗样本以递增混合比例注入训练集，逐步强化零样本的RoBERTa-base模型。在标准基准测试中，VAULT使RoBERTa-base在SNLI上的准确率从88.48%提升至92.60%（+4.12%），在ANLI上从75.04%提升至80.95%（+5.91%），在MultiNLI上从54.67%提升至71.99%（+17.32%）。该方法在所有数据集上均优于现有上下文对抗方法，最高提升达2.0%。通过大规模自动化高质量对抗数据构建，VAULT实现了无需人工干预的NLI推理任务鲁棒性快速增强。
原文摘要 · Abstract (English)
We introduce VAULT, a fully automated adversarial RAG pipeline that systematically uncovers and remedies weaknesses in NLI models through three stages: retrieval, adversarial generation, and iterative retraining. First, we perform balanced few-shot retrieval by embedding premises with both semantic (BGE) and lexical (BM25) similarity. Next, we assemble these contexts into LLM prompts to generate adversarial hypotheses, which are then validated by an LLM ensemble for label fidelity. Finally, the validated adversarial examples are injected back into the training set at increasing mixing ratios, progressively fortifying a zero-shot RoBERTa-base model.On standard benchmarks, VAULT elevates RoBERTa-base accuracy from 88.48% to 92.60% on SNLI +4.12%, from 75.04% to 80.95% on ANLI +5.91%, and from 54.67% to 71.99% on MultiNLI +17.32%. It also consistently outperforms prior in-context adversarial methods by up to 2.0% across datasets. By automating high-quality adversarial data curation at scale, VAULT enables rapid, human-independent robustness improvements in NLI inference tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。