用图结构提升大模型推理能力，解决传统文本推理的评估难题。
Graph Reasoning Paradigm: Structured and Symbolic Reasoning with Topology-Aware Reinforcement Learning for Large Language Models
- 将推理过程转为带认知标签的图结构，实现结构化表达。
- 在数学和代码任务上显著提升性能，优于现有方法。
- 适合研究大模型推理机制与强化学习优化的学者使用。
长链式思维（LCoT）通过可验证奖励的强化学习（RLVR）已被证明能有效增强大语言模型（LLM）的推理能力。然而，当前LLM的推理仍以纯文本形式生成，对这类非结构化数据进行语义评估会带来训练时的计算瓶颈。尽管采用RLVR优化，现有方法仍存在监督粒度粗、奖励欺骗、训练成本高及泛化性差等问题。为此，我们提出图推理范式（GRP），通过带有步骤级认知标签的图结构表示，实现结构化与符号化推理。在此基础上，设计了过程感知分层裁剪组相对策略优化（PASC-GRPO），利用结构化评估替代语义评估，通过图结构结果奖励实现过程感知验证，并基于分层裁剪优势估计缓解奖励欺骗。实验表明，在数学推理与代码生成任务中均有显著提升。数据、模型与代码将后续发布。
原文摘要 · Abstract (English)
Long Chain-of-Thought (LCoT), achieved by Reinforcement Learning with Verifiable Rewards (RLVR), has proven effective in enhancing the reasoning capabilities of Large Language Models (LLMs). However, reasoning in current LLMs is primarily generated as plain text, where performing semantic evaluation on such unstructured data creates a computational bottleneck during training. Despite RLVR-based optimization, existing methods still suffer from coarse-grained supervision, reward hacking, high training costs, and poor generalization. To address these issues, we propose the Graph Reasoning Paradigm (GRP), which realizes structured and symbolic reasoning, implemented via graph-structured representations with step-level cognitive labels. Building upon GRP, we further design Process-Aware Stratified Clipping Group Relative Policy Optimization (PASC-GRPO), which leverages structured evaluation to replace semantic evaluation, achieves process-aware verification through graph-structured outcome rewards, and mitigates reward hacking via stratified clipping advantage estimation. Experiments demonstrate significant improvements across mathematical reasoning and code generation tasks. Data, models, and code will be released later.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。