arXiv:2510.15191cs.CLcs.AI2025-10被引 5

让大模型动态生成结构化知识,提升推理准确率。

Structure-R1: Dynamically Leveraging Structural Knowledge in LLM Reasoning through Reinforcement Learning

  • 用强化学习生成任务适配的结构化知识表示。
  • 在70亿参数模型上达到超大模型水平的推理性能。
  • 适合需要精准知识推理的科研与工程场景。

大语言模型在推理能力上取得显著进展,但受限于对显式结构化领域知识的访问不足。检索增强生成(RAG)通过引入外部信息来弥补这一缺陷,但传统RAG系统通常处理非结构化、碎片化的文本,导致信息密度低且推理效果不佳。为此,我们提出 extsc{Structure-R1},一种将检索内容转化为优化推理的结构化表示的新框架。该框架利用强化学习,学习一种内容表征策略,能根据多步推理需求动态生成并调整结构格式。不同于依赖固定模式的方法,我们的方法采用生成式范式,可为每个查询定制特定结构。为确保表示质量,引入自奖励结构验证机制,检验生成结构是否正确且自洽。在七个知识密集型基准上的实验表明, extsc{Structure-R1}使用7B规模主干模型即可实现竞争力表现,并媲美更大模型。理论分析进一步说明,结构化表示通过提升信息密度和上下文清晰度增强推理能力。代码与数据已公开:https://github.com/jlwu002/sr1。

原文摘要 · Abstract (English)

Large language models (LLMs) have demonstrated remarkable advances in reasoning capabilities. However, their performance remains constrained by limited access to explicit and structured domain knowledge. Retrieval-Augmented Generation (RAG) addresses this by incorporating external information as context to augment reasoning. Nevertheless, traditional RAG systems typically operate over unstructured and fragmented text, resulting in low information density and suboptimal reasoning. To overcome these limitations, we propose \textsc{Structure-R1}, a novel framework that transforms retrieved content into structured representations optimized for reasoning. Leveraging reinforcement learning, \textsc{Structure-R1} learns a content representation policy that dynamically generates and adapts structural formats based on the demands of multi-step reasoning. Unlike prior methods that rely on fixed schemas, our approach adopts a generative paradigm capable of producing task-specific structures tailored to individual queries. To ensure the quality and reliability of these representations, we introduce a self-reward structural verification mechanism that checks whether the generated structures are both correct and self-contained. Extensive experiments on seven knowledge-intensive benchmarks show that \textsc{Structure-R1} consistently achieves competitive performance with a 7B-scale backbone model and matches the performance of much larger models. Additionally, our theoretical analysis demonstrates how structured representations enhance reasoning by improving information density and contextual clarity. Our code and data are available at: https://github.com/jlwu002/sr1.

大模型推理结构化知识强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。