arXiv:2608.01534cs.CV2026-08

用递归机制提升大模型解符号谜题能力,效果显著优于传统方法。

Recursive Vision Language Models for General Symbolic Reasoning

论文配图:Recursive Vision Language Models for General Symbolic Reasoning
图 1 · 摘自论文原文
  • 基于预训练模型构建递归推理框架,支持反复修正答案。
  • 在ARC-AGI上比基线提升27.6%,八项任务均表现领先。
  • 适合需要逻辑推理与迭代优化的研究者和开发者。

Sudoku、迷宫寻路和ARC等硬性符号推理任务对大语言模型仍具挑战,因其固定深度的自回归推理限制了系统性搜索、修正与回溯。尽管递归模型如HRM和TRM通过迭代隐状态修正缓解此问题,但通常任务特定且未利用预训练语言先验。本文提出R-Qwen,基于预训练Qwen架构的递归推理框架,通过程序化自我递归和深度监督反复优化候选解,融合递归模型的结构化迭代计算与预训练模型的语言及推理先验。我们进一步将分层监督加权(HSW)引入自回归模型,指数加权各递归步损失,使梯度方差降低至少50%,提升梯度信噪比并加速收敛。在八项挑战性基准上,R-Qwen持续优于先前递归模型,并显著超越参数量更大的大模型,仅使用相近可训练参数量。尤其在ARC-AGI数据集上,较基线提升27.6%,凸显递归精炼对通用符号推理的有效性。结果表明,递归推理机制与预训练语言模型先验是互补的符号谜题求解路径。代码与模型将在论文录用后发布。

原文摘要 · Abstract (English)

Hard symbolic-reasoning tasks such as Sudoku, maze pathfinding, and ARC remain challenging for LLMs due to their fixed-depth autoregressive reasoning, which limits systematic search, refinement, and backtracking. While recursive models such as Hierarchical Reasoning Model (HRM) and Tiny Recursive Model (TRM) address this limitation through iterative latent-state refinement, they are typically task-specific and do not leverage pretrained language priors. We propose R-Qwen, a recursive reasoning framework built upon a pretrained Qwen backbone. R-Qwen repeatedly refines a candidate solution through programmatic self-recursion and deep supervision, combining the structured iterative computation of recursive models with the linguistic and reasoning priors of pretrained LLMs. We further adapt Hierarchical Supervision Weighting (HSW) to autoregressive models by exponentially weighting losses across recursive steps. HSW reduces gradient variance by at least 50\%, improves the signal-to-noise ratio of stochastic gradients, and accelerates convergence. Across eight challenging benchmarks, R-Qwen consistently outperforms prior recursive reasoning models and substantially larger LLMs while using a comparable number of trainable parameters. Notably, on ARC-AGI dataset, our model achieves a 27.6\% improvement over the baseline, highlighting the effectiveness of recursive refinement for general symbolic reasoning. These results suggest that recursive reasoning mechanisms and pretrained language model priors are complementary approaches for improving symbolic puzzle-solving. Code and models will be released after acceptance.

符号推理递归模型大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。