arXiv:2505.22848cs.CL2025-05EMNLP被引 5

构建解释分类体系,揭示同标签下人类推理差异。

LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inference

  • 提出语言学驱动的解释分类体系LITEx,系统归类自由文本解释。
  • 在e-SNLI数据集上验证分类可靠性,发现同标签下解释差异显著。
  • 用分类指导生成解释,使模型输出更贴近人类推理逻辑。

自然语言推理(NLI)中存在人类标注者对同一前提-假设对给出不同标签的现象(人类标签变异,HLV)。然而,标注者虽一致打相同标签但理由迥异的“同标签差异”问题却鲜受关注。部分NLI数据集标注了关键词作为解释,但同一词段可因不同原因被标注,自由文本解释能揭示此差异。为此,本文提出LITEx:一种基于语言学的英文解释分类体系。利用该体系,我们对e-SNLI子集进行标注,验证了分类的可靠性和一致性,并分析其与标签、高亮词及解释之间的关系。进一步评估发现,以LITEx为条件生成解释,其语言特征更接近人类解释,优于仅依赖标签或高亮词的方法。本方法不仅捕捉同标签下的推理多样性,还表明分类引导生成能更好弥合人类与模型解释的差距。

原文摘要 · Abstract (English)

There is increasing evidence of Human Label Variation (HLV) in Natural Language Inference (NLI), where annotators assign different labels to the same premise-hypothesis pair. However, within-label variation--cases where annotators agree on the same label but provide divergent reasoning--poses an additional and mostly overlooked challenge. Several NLI datasets contain highlighted words in the NLI item as explanations, but the same spans on the NLI item can be highlighted for different reasons, as evidenced by free-text explanations, which offer a window into annotators' reasoning. To systematically understand this problem and gain insight into the rationales behind NLI labels, we introduce LITEX, a linguistically-informed taxonomy for categorizing free-text explanations in English. Using this taxonomy, we annotate a subset of the e-SNLI dataset, validate the taxonomy's reliability, and analyze how it aligns with NLI labels, highlights, and explanations. We further assess the taxonomy's usefulness in explanation generation, demonstrating that conditioning generation on LITEX yields explanations that are linguistically closer to human explanations than those generated using only labels or highlights. Our approach thus not only captures within-label variation but also shows how taxonomy-guided generation for reasoning can bridge the gap between human and model explanations more effectively than existing strategies.

自然语言推理解释生成标注差异

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。