用大模型生成反事实数据，提升文本推理模型可靠性
Mitigating Spurious Correlations in NLI via LLM-Synthesized Counterfactuals and Dynamic Balanced Sampling
- 通过大模型合成反事实样本，自动构建高质量对比数据集
- 在挑战性基准上推理一致性从63.5%提升至81.0%
- 适合关注模型鲁棒性与可解释性的研究者
自然语言推理（NLI）模型常依赖表面相关性而非语义推理。现有缓解方法往往成本高或导致微调时灾难性遗忘。我们提出一种自动化、可扩展的解决方案：首先引入对数频率LMI（LF-LMI）精准检测语义伪特征；其次通过大模型合成管道结合多评审验证生成高质量合成对比集；最后提出动态平衡采样策略，通过轮换原始数据分布防止遗忘。该方法在挑战性基准上将推理一致性从63.5%提升至81.0%，同时保持88.4%的域内准确率，显著优于简单微调。
原文摘要 · Abstract (English)
Natural Language Inference (NLI) models frequently rely on spurious correlations rather than semantic reasoning. Existing mitigation strategies often incur high annotation costs or trigger catastrophic forgetting during fine-tuning. We propose an automated, scalable pipeline to address these limitations. First, we introduce Log-Frequency LMI (LF-LMI) to accurately detect semantic artifacts. Second, we generate a high-quality synthetic contrast set via an LLM-synthesis pipeline with multi-judge verification. Finally, we introduce Dynamic Balanced Sampling, a training strategy that rotates the original data distribution to prevent forgetting. Our method improves consistency on a challenging benchmark from 63.5% to 81.0% while maintaining 88.4% in-domain accuracy, significantly outperforming naive fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。