用多步去噪递归提升推理能力,让模型更聪明地逐步修正答案。
One Step Forward and K Steps Back: Better Reasoning with Denoising Recursion Models

- 通过多步递归去噪,让模型分步修正输出,而非单步跳跃。
- 在ARC-AGI任务上超越旧方法,实现突破性表现。
- 适合需要深度思考和复杂推理的AI研究者使用。
循环变压器通过重复应用共享的Transformer块,在不增加参数量的情况下扩展计算深度,可用于迭代优化,每轮并行重写整个固定大小的预测结果。对于需要类似搜索的复杂问题,从噪声开始生成高度结构化的解可能需要漫长的优化路径。当训练仅指定最终目标而无中间路径监督时,学习这些路径极具挑战。扩散模型通过在不同噪声水平下污染数据并训练模型进行单步逆向恢复来应对,但其训练与测试行为不一致。本文提出去噪递归模型(Denoising Recursion Models),同样对数据施加噪声,但训练模型在多个递归步骤中逐步逆转噪声。该策略提供可管理的中间状态课程,更匹配训练与测试行为,并激励非贪婪、前瞻性的生成。大量实验表明,该方法在ARC-AGI上优于小规模递归模型(TRM),实现了突破性性能。
原文摘要 · Abstract (English)
Looped transformers scale computational depth without increasing parameter count by repeatedly applying a shared transformer block and can be used for iterative refinement, where each loop rewrites a full fixed-size prediction in parallel. On difficult problems, such as those that require search-like computation, reaching a highly structured solution starting from noise can require long refinement trajectories. Learning such trajectories is challenging when training specifies only the target solution and provides no supervision over the intermediate refinement path. Diffusion models tackle this issue by corrupting data with varying magnitudes of noise and training the model to reverse it in a \textit{single step}. However, this process misaligns training and testing behaviour. We introduce Denoising Recursion Models, a method that similarly corrupts data with noise but trains the model to reverse the corruption over \textit{multiple} recursive steps. This strategy provides a tractable curriculum of intermediate states, while better aligning training with testing and incentivizing non-greedy, forward-looking generation. Through extensive experiments, we show this approach outperforms the Tiny Recursion Model (TRM) on ARC-AGI, where it recently achieved breakthrough performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。