arXiv:2506.15211cs.CL2025-06被引 9

用抽象推理原型提升大模型跨领域推理能力

ProtoReasoning: Prototypes as the Foundation for Generalizable Reasoning in LLMs

  • 构建可验证的原型表示,将问题转化为通用推理模式
  • 在逻辑、规划等任务上分别提升4.7%至6.3%
  • 适合研究大模型泛化机制与可解释推理的学者

近期基于长链式思维(Long CoT)训练的大规模推理模型展现出出色的跨领域泛化能力,但其内在机制仍不明确。我们假设跨领域泛化源于共享的抽象推理原型——即捕捉不同领域问题本质的通用推理模式,这些原型通过最小化表征细节,揭示看似多样的任务实则共享推理结构。基于此,我们提出ProtoReasoning框架,通过可扩展且可验证的原型表示(如Prolog用于逻辑推理,PDDL用于规划)增强大语言模型的推理能力。该框架包含:(1) 自动化原型构建流程,将问题转换为对应原型表示;(2) 基于Prolog/PDDL解释器的全面验证系统,提供可靠反馈;(3) 在原型空间中任意合成问题并保证正确性的可扩展性。大量实验表明,ProtoReasoning在逻辑推理(Enigmata-Eval)上比基线提升4.7%,规划任务提升6.3%,通用推理(MMLU)提升4.0%,数学(AIME24)提升1.0%。显著的是,消融实验确认,在原型空间中学习相比仅使用自然语言表示,对结构相似问题具有更强泛化能力,验证了推理原型是大语言模型可泛化推理的基础。

原文摘要 · Abstract (English)

Recent advances in Large Reasoning Models (LRMs) trained with Long Chain-of-Thought (Long CoT) reasoning have demonstrated remarkable cross-domain generalization capabilities. However, the underlying mechanisms supporting such transfer remain poorly understood. We hypothesize that cross-domain generalization arises from shared abstract reasoning prototypes -- fundamental reasoning patterns that capture the essence of problems across domains. These prototypes minimize the nuances of the representation, revealing that seemingly diverse tasks are grounded in shared reasoning structures.Based on this hypothesis, we propose ProtoReasoning, a framework that enhances the reasoning ability of LLMs by leveraging scalable and verifiable prototypical representations (Prolog for logical reasoning, PDDL for planning).ProtoReasoning features: (1) an automated prototype construction pipeline that transforms problems into corresponding prototype representations; (2) a comprehensive verification system providing reliable feedback through Prolog/PDDL interpreters; (3) the scalability to synthesize problems arbitrarily within prototype space while ensuring correctness. Extensive experiments show that ProtoReasoning achieves 4.7% improvement over baseline models on logical reasoning (Enigmata-Eval), 6.3% improvement on planning tasks, 4.0% improvement on general reasoning (MMLU) and 1.0% on mathematics (AIME24). Significantly, our ablation studies confirm that learning in prototype space also demonstrates enhanced generalization to structurally similar problems compared to training solely on natural language representations, validating our hypothesis that reasoning prototypes serve as the foundation for generalizable reasoning in large language models.

大模型推理原型表示可解释性泛化能力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。