用迭代反馈提升大模型生成反事实样本的准确性
iFlip: Iterative Feedback-driven Counterfactual Example Refinement
- 通过模型置信度、特征归因和自然语言三类反馈迭代优化反事实样本
- 反事实有效率比现有方法平均高出57.8%，标签翻转率显著提升
- 适合需要可解释性与数据增强的研究者，尤其适用于NLP场景
反事实样本是对输入进行最小修改以改变模型预测结果的实例，广泛应用于可解释人工智能和自然语言处理中的训练数据增强。然而，使用大语言模型生成有效的反事实样本仍具挑战性，现有单次生成方法常无法引发可靠标签变化，忽视了大模型的自我修正能力。为此，我们提出iFlip，一种基于迭代反馈的反事实样本精炼方法，利用模型置信度、特征归因和自然语言三类反馈。实验表明，iFlip在标签翻转率上比五种最先进基线平均提高57.8%。用户研究进一步验证了iFlip在完整性、满意度和可行性方面优于基线。消融实验显示，三个关键因素对生成有效反事实至关重要：适当的迭代次数、聚焦高归因词、以及早期停止策略。此外，iFlip生成的反事实样本可用于有效数据增强，显著提升模型性能与鲁棒性。
原文摘要 · Abstract (English)
Counterfactual examples are minimal edits to an input that alter a model's prediction. They are widely employed in explainable AI to probe model behavior and in natural language processing (NLP) to augment training data. However, generating valid counterfactuals with large language models (LLMs) remains challenging, as existing single-pass methods often fail to induce reliable label changes, neglecting LLMs' self-correction capabilities. To explore this untapped potential, we propose iFlip, an iterative refinement approach that leverages three types of feedback, including model confidence, feature attribution, and natural language. Our results show that iFlip achieves an average 57.8% higher validity than the five state-of-the-art baselines, as measured by the label flipping rate. The user study further corroborates that iFlip outperforms baselines in completeness, overall satisfaction, and feasibility. In addition, ablation studies demonstrate that three components are paramount for iFlip to generate valid counterfactuals: leveraging an appropriate number of iterations, pointing to highly attributed words, and early stopping. Finally, counterfactuals generated by iFlip enable effective counterfactual data augmentation, substantially improving model performance and robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。