arXiv:2508.10019cs.CLcs.AI2025-08AAAI被引 2

将理解与推理解耦,让小模型更高效地解决数学逻辑题。

Decoupling Understanding from Reasoning via Problem Space Mapping for Small-Scale Model Reasoning

  • 用强化学习把自然语言问题映射到标准化语义空间。
  • 在标准空间中训练小模型,推理准确率显著提升。
  • 适合资源有限但需强推理能力的场景,如移动端部署。

尽管大型语言模型(LLMs)在推理能力上取得进展,但提升小型语言模型(SLMs,如参数量达1.5B)的推理能力仍具挑战。主要障碍在于自然语言的复杂性与多样性:本质相同的问题常以不同表面形式呈现,被冗余或干扰信息掩盖。这给容量有限的SLMs带来双重负担:需先从复杂输入中提取核心问题,再基于理解进行推理。由此产生的庞大且嘈杂的问题空间阻碍优化。为此,我们提出新框架,通过将自然语言问题映射至规范问题空间——一个语义简化但表达力强的领域,实现理解与推理的解耦。在此框架下,我们设计DURIT(Decoupled Understanding from Reasoning via Iterative Training),一种三步迭代算法:(1) 通过强化学习映射自然语言问题;(2) 利用自蒸馏对齐推理轨迹;(3) 在问题空间中训练推理策略。映射器与推理器在过程中交替联合训练。实验表明,DURIT显著提升SLMs在数学与逻辑推理任务上的表现,涵盖域内与域外数据。除增强推理能力外,也提升了推理鲁棒性,验证了理解-推理解耦是强化小模型的有效策略。

原文摘要 · Abstract (English)

Despite recent advances in the reasoning capabilities of Large Language Models (LLMs), improving the reasoning ability of Small Language Models (SLMs, e.g., up to 1.5B parameters) remains challenging. A key obstacle lies in the complexity and variability of natural language: essentially equivalent problems often appear in diverse surface forms, often obscured by redundant or distracting details. This imposes a dual burden on SLMs: they must first extract the core problem from complex linguistic input, and then perform reasoning based on that understanding. The resulting vast and noisy problem space hinders optimization, particularly for models with limited capacity. To address this, we propose a new framework that decouples understanding from reasoning by mapping natural language problems into a canonical problem space-a semantically simplified yet expressive domain. This enables SLMs to focus on reasoning over standardized inputs, free from linguistic variability. Within this framework, we introduce DURIT (Decoupled Understanding from Reasoning via Iterative Training), a three-step algorithm that iteratively: (1) mapping natural language problems via reinforcement learning, (2) aligns reasoning trajectories through self-distillation, and (3) trains reasoning policies in the problem space. The mapper and reasoner are co-trained in an alternating loop throughout this process. Experiments show that DURIT substantially improves SLMs' performance on both in-domain and out-of-domain mathematical and logical reasoning tasks. Beyond improving reasoning capabilities, DURIT also improves the robustness of reasoning, validating decoupling understanding from reasoning as an effective strategy for strengthening SLMs.

小模型推理解耦强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。