通过生成反事实样本提升模型对常识性语句的判断能力
Counterfactual Samples Constructing and Training for Commonsense Statements Estimation
- 用关键词替换和低层丢弃生成反事实句子
- 在九个数据集上比现有最优方法提升3.07%
- 让模型关注关键词汇，更敏感地捕捉常识差异
可读性评估（PE）对语言模型客观理解现实世界至关重要。尽管大语言模型在PE任务中表现优异，但因常识知识复杂，仍常出现简单常识错误。现有模型缺乏两个关键特性：一是语言可解释性——决策依赖关键词片段；二是常识敏感性——能察觉常识性语言细微变化。为此，我们提出一种模型无关的新方法——常识反事实样本生成（CCSG）。通过在训练中引入CCSG生成的反事实样本，引导模型聚焦关键词汇，从而增强其语言可解释性和常识敏感性。具体而言，CCSG通过策略性替换关键词并引入低层随机丢弃生成反事实样本，并将其融入句子级对比学习框架，强化模型学习。在九个不同数据集上的实验表明，该方法有效应对常识推理挑战，相较当前最优方法提升3.07%。
原文摘要 · Abstract (English)
Plausibility Estimation (PE) plays a crucial role for enabling language models to objectively comprehend the real world. While large language models (LLMs) demonstrate remarkable capabilities in PE tasks but sometimes produce trivial commonsense errors due to the complexity of commonsense knowledge. They lack two key traits of an ideal PE model: a) Language-explainable: relying on critical word segments for decisions, and b) Commonsense-sensitive: detecting subtle linguistic variations in commonsense. To address these issues, we propose a novel model-agnostic method, referred to as Commonsense Counterfactual Samples Generating (CCSG). By training PE models with CCSG, we encourage them to focus on critical words, thereby enhancing both their language-explainable and commonsense-sensitive capabilities. Specifically, CCSG generates counterfactual samples by strategically replacing key words and introducing low-level dropout within sentences. These counterfactual samples are then incorporated into a sentence-level contrastive training framework to further enhance the model's learning process. Experimental results across nine diverse datasets demonstrate the effectiveness of CCSG in addressing commonsense reasoning challenges, with our CCSG method showing 3.07% improvement against the SOTA methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。