让大模型学会跨表示形式理解流程，提升真实场景应用能力
Can Large Language Models Generalize Procedures Across Representations?
- 用两阶段强化学习课程，先学符号化数据再学自然语言
- 1.5B参数模型经训练后接近零样本GPT-4o在规划任务表现
- 发现跨表示泛化本质是生成类比，可被课程设计有效激发
大型语言模型（LLMs）在代码、图等符号化表示上训练和测试广泛，但现实用户任务常以自然语言描述。模型能否在不同表示间泛化？我们研究了代码、图与自然语言中同构的流程任务（如计划调度）。结果表明，仅在图或代码数据上训练无法可靠迁移到自然语言任务；仅在自然语言上训练则效率低下。为此，我们提出两阶段强化学习课程：先训练于符号化数据，再过渡到自然语言。该方法显著提升多种模型家族与任务的表现。令人惊讶的是，使用该方法训练的1.5B参数Qwen模型，在自然语言规划任务上接近零样本GPT-4o水平。分析显示，成功的跨表示泛化可视为生成类比，而我们的课程能有效促进这一机制。相关数据集与代码见GitHub。
原文摘要 · Abstract (English)
Large language models (LLMs) are trained and tested extensively on symbolic representations such as code and graphs, yet real-world user tasks are often specified in natural language. To what extent can LLMs generalize across these representations? Here, we approach this question by studying isomorphic tasks involving procedures represented in code, graphs, and natural language (e.g., scheduling steps in planning). We find that training LLMs with popular post-training methods on graphs or code data alone does not reliably generalize to corresponding natural language tasks, while training solely on natural language can lead to inefficient performance gains. To address this gap, we propose a two-stage reinforcement learning curriculum that first trains on symbolic, then natural language data. The curriculum substantially improves model performance across model families and tasks. Remarkably, a 1.5B Qwen model trained by our method can closely match zero-shot GPT-4o in naturalistic planning. Finally, our analysis suggests that successful cross-representation generalization can be interpreted as a form of generative analogy, which our curriculum effectively encourages. The dataset and code used in this paper can be found \href{https://github.com/fangru-lin/procedure_generalization_llm}{here}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。