让大模型学会对称与反对称关系，提升逻辑推理能力
Capturing Symmetry and Antisymmetry in Language Models through Symmetry-Aware Training Objectives
- 用维基数据构建自然语言推理数据集，评估模型关系理解
- 原生大模型在对称/反对称关系上表现仅如随机猜测
- 对比学习重训练编码器，提升少样本学习与遗忘缓解
捕捉对称关系（如国家接壤）和反对称关系（如父母关系）对于多种应用至关重要。本文提出一个基于维基数据的自然语言推理数据集，用于评估大语言模型（LLMs）的这一能力。实验发现，现有大模型在此基准上的表现接近随机水平，暴露出其在关系理解上的显著不足。为解决该问题，我们采用基于k近邻的对比学习对编码器进行重训练。结果显示，重训练后的编码器性能可媲美微调分类头，同时具备更高效的少样本学习能力以及更强的灾难性遗忘抑制效果。
原文摘要 · Abstract (English)
Capturing symmetric (e.g., country borders another country) and antisymmetric (e.g., parent_of) relations is crucial for a variety of applications. This paper tackles this challenge by introducing a novel Wikidata-derived natural language inference dataset designed to evaluate large language models (LLMs). Our findings reveal that LLMs perform comparably to random chance on this benchmark, highlighting a gap in relational understanding. To address this, we explore encoder retraining via contrastive learning with k-nearest neighbors. The retrained encoder matches the performance of fine-tuned classification heads while offering additional benefits, including greater efficiency in few-shot learning and improved mitigation of catastrophic forgetting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。