模型缺乏隐式多跳推理能力，根源在于预训练时未接触组合性上下文。
Multi-Hop Knowledge Composition is Bound by Pretraining Exposure
- 通过分离预训练中是否出现组合性上下文，控制实验变量。
- 97%单跳准确率下仍无法完成多跳推理，证明是预训练缺陷而非知识缺失。
- 仅在预训练中接触过组合性上下文的个体才能实现跨事实推理。
大型语言模型在隐式多跳推理上表现失败：即使能正确回答“X何时出生？”和“Y最亲密的朋友是谁？”，也无法在单次前向传播中回答“Y最亲密的朋友何时出生？”，尽管这两个事实均被完全记忆且可独立检索。我们在受控自然语言场景中研究此现象，严格区分预训练中曾暴露于组合性上下文的个体与从未出现在此类上下文中的个体。结果表明，即使单跳准确率达97%，组合推理失败依然存在，说明该问题源于预训练阶段而非知识缺失。我们测试了九种数据增强格式，发现只有在预训练中接触过组合性上下文的个体，其组合推理能力才能迁移到未见问题；而从未接触过的个体则始终无法实现。这表明，预训练期间接触组合性上下文是隐式多跳推理的必要条件。
原文摘要 · Abstract (English)
Large Language Models fail at implicit multi-hop reasoning: a model answers "When was $X$ born?" and "Who is $Y$'s closest friend?" correctly but fails on "When was $Y$'s closest friend born?" in a single forward pass, even when both facts are perfectly memorized and individually retrievable. We study this failure in a controlled natural language setting with a strict separation between individuals exposed to compositional contexts during pretraining and those that never appear in any such context. We confirm that compositional failure persists even at 97% 1-hop accuracy, establishing the gap as a pretraining failure rather than a knowledge absence. We propose and test nine data-centric augmentation formats and find that compositional pretraining transfers to unseen questions for exposed individuals, but never to individuals absent from compositional pretraining, suggesting that exposure to compositional contexts during pretraining is a necessary condition for implicit multi-hop reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。