为理解模型在否定查询上的表现短板，提出系统性分类框架与评测数据集。
A Comprehensive Taxonomy of Negation for NLP and Neural Retrievers
- 基于哲学语言学构建否定类型分类体系
- 创建两个基准数据集，提升模型对否定的处理能力
- 揭示现有数据覆盖缺陷，指导模型优化方向
理解并解决复杂推理任务对满足用户信息需求至关重要。尽管密集神经模型能学习上下文嵌入，但在包含否定的查询上仍表现不佳。为此，我们研究了传统神经信息检索和基于大模型的检索中否定现象。首先，提出一个源自哲学、语言学和逻辑定义的否定分类体系；其次，构建两个可用于评估神经检索模型性能并用于微调以增强否定鲁棒性的基准数据集；最后，提出一种基于逻辑的分类机制，用于分析检索模型在现有数据集上的表现。该分类体系实现了否定类型间的平衡数据分布，使模型在NevIR数据集上收敛速度更快。此外，该分类方案揭示了现有数据集中否定类型的覆盖情况，为理解微调模型在否定任务上的泛化能力影响因素提供了洞见。
原文摘要 · Abstract (English)
Understanding and solving complex reasoning tasks is vital for addressing the information needs of a user. Although dense neural models learn contextualised embeddings, they still underperform on queries containing negation. To understand this phenomenon, we study negation in both traditional neural information retrieval and LLM-based models. We (1) introduce a taxonomy of negation that derives from philosophical, linguistic, and logical definitions; (2) generate two benchmark datasets that can be used to evaluate the performance of neural information retrieval models and to fine-tune models for a more robust performance on negation; and (3) propose a logic-based classification mechanism that can be used to analyze the performance of retrieval models on existing datasets. Our taxonomy produces a balanced data distribution over negation types, providing a better training setup that leads to faster convergence on the NevIR dataset. Moreover, we propose a classification schema that reveals the coverage of negation types in existing datasets, offering insights into the factors that might affect the generalization of fine-tuned models on negation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。