测试Transformer能否拼接训练数据中的知识进行推理
Are Transformers Able to Reason by Connecting Separated Knowledge in Training Data?
- 设计碎片化训练、链式测试任务，模拟人类组合推理
- 少样本思维链提示让模型成功拼出未在训练中出现的因果链
- 模型复杂度和数据相似性决定推理能力是否涌现
人类能通过整合不同来源的知识进行组合推理。例如，若从一处学到 ( B = f(A) )，从另一处学到 ( C = g(B) )，即使从未见过 ( ABC ) 同时出现，也能推导出 ( C=g(f(A)) )，体现强大的泛化能力。本文提出一种合成学习任务「FTCT」（训练时碎片化，测试时链式连接），验证Transformer复制此能力的潜力，并解析其内在机制。训练阶段，数据由整体因果图的分离知识片段构成；测试阶段，模型需通过整合这些片段推断完整因果路径。结果表明，少量样本的思维链提示使Transformer能在测试中正确组合未在训练中出现的片段，实现组合推理。此外，组合推理能力的出现与模型复杂度及训练-测试数据相似性高度相关。我们从理论和实证两方面提出，Transformer在训练中学习到一个可泛化的底层程序，从而在测试中实现有效组合推理。
原文摘要 · Abstract (English)
Humans exhibit remarkable compositional reasoning by integrating knowledge from various sources. For example, if someone learns ( B = f(A) ) from one source and ( C = g(B) ) from another, they can deduce ( C=g(B)=g(f(A)) ) even without encountering ( ABC ) together, showcasing the generalization ability of human intelligence. In this paper, we introduce a synthetic learning task, "FTCT" (Fragmented at Training, Chained at Testing), to validate the potential of Transformers in replicating this skill and interpret its inner mechanism. In the training phase, data consist of separated knowledge fragments from an overall causal graph. During testing, Transformers must infer complete causal graph traces by integrating these fragments. Our findings demonstrate that few-shot Chain-of-Thought prompting enables Transformers to perform compositional reasoning on FTCT by revealing correct combinations of fragments, even if such combinations were absent in the training data. Furthermore, the emergence of compositional reasoning ability is strongly correlated with the model complexity and training-testing data similarity. We propose, both theoretically and empirically, that Transformers learn an underlying generalizable program from training, enabling effective compositional reasoning during testing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。