arXiv:2503.22362cs.CL2025-03中稿 · COLM被引 8

模型对等价事实识别不均,源于训练数据中实体出现频率差异。

Supposedly Equivalent Facts That Aren't? Entity Frequency in Pre-training Induces Asymmetry in LLMs

论文配图:Supposedly Equivalent Facts That Aren't? Entity Frequency in Pre-training Induces Asymmetry in LLMs
图 1 · 摘自论文原文
  • 通过分析知识三元组,发现高频主语+低频宾语事实更易被识别
  • 高频主语与低频宾语组合的识别率比反向高出18.3%
  • 适用于研究模型幻觉成因及闭源模型数据特征推断

理解并缓解大语言模型(LLMs)中的幻觉问题对确保可靠内容生成至关重要。以往研究多关注模型何时产生幻觉,而本文揭示了其背后原因,并直接将模型行为与预训练数据中的先验知识关联。我们发现,逻辑等价的事实在识别上存在不对称性,这可归因于实体作为主语或宾语时的频率差异。由于多数预训练数据集不可访问,我们利用完全开源的OLMo系列及其索引的Dolma数据集估算实体频率。基于Wikidata5M中的关系事实(三元组),构建探针数据集以隔离该效应。实验表明,在主语高频、宾语低频的情况下,事实识别率显著高于其逆向情况;而在低频到高频转换时,该模式反转;当两个实体均为高频时,无统计显著不对称性。这些发现凸显了预训练数据对模型预测的深远影响,为推断闭源或部分闭源模型的训练数据特征提供了新思路。

原文摘要 · Abstract (English)

Understanding and mitigating hallucinations in Large Language Models (LLMs) is crucial for ensuring reliable content generation. While previous research has primarily focused on "when" LLMs hallucinate, our work explains "why" and directly links model behaviour to the pre-training data that forms their prior knowledge. Specifically, we demonstrate that an asymmetry exists in the recognition of logically equivalent facts, which can be attributed to frequency discrepancies of entities appearing as subjects versus objects. Given that most pre-training datasets are inaccessible, we leverage the fully open-source OLMo series by indexing its Dolma dataset to estimate entity frequencies. Using relational facts (represented as triples) from Wikidata5M, we construct probing datasets to isolate this effect. Our experiments reveal that facts with a high-frequency subject and a low-frequency object are better recognised than their inverse, despite their logical equivalence. The pattern reverses in low-to-high frequency settings, and no statistically significant asymmetry emerges when both entities are high-frequency. These findings highlight the influential role of pre-training data in shaping model predictions and provide insights for inferring the characteristics of pre-training data in closed or partially closed LLMs.

大模型幻觉预训练数据知识推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。