arXiv:2605.21488cs.LG2026-05被引 8

通过学习任务相关吸引子,实现可扩展的推理能力。

Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning

论文配图:Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning
图 1 · 摘自论文原文
  • 用吸引子机制替代传统迭代推理,让模型自适应分配计算资源。
  • 在Sudoku-Extreme上,准确率从2.6%提升至99%以上,等效40,000层推理。
  • 适合需要高精度、可扩展推理的任务,如复杂逻辑题和规划问题。

通过在测试时迭代更新隐状态来扩展计算量,已成为一种强大的推理范式。然而,这些迭代模型如何泛化于记忆模式之外仍不清晰。我们提出,可泛化的推理源于学习任务相关的吸引子:其稳定不动点对应有效解的隐动态系统。我们通过等平衡推理器(EqR)形式化该过程,无需外部验证器或任务特定先验即可实现测试时扩展。EqR沿两个维度扩展内部动态:深度(更多迭代步数)与广度(多初始值生成随机轨迹并聚合)。实证表明,测试时扩展带来的增益与向解对齐吸引子的强收敛性紧密耦合。该吸引子视角使神经网络能根据任务难度自适应分配测试时计算资源:简单任务1~5步即收敛,复杂任务则受益于大规模测试时扩展。通过展开相当于40,000层的推理,可扩展隐状态推理将准确率从前馈模型的2.6%提升至超过99%(在Sudoku-Extreme上)。结果表明,学习到的吸引子景观为理解迭代隐状态模型中的可扩展推理提供了有效的机制视角。

原文摘要 · Abstract (English)

Scaling test-time compute by iteratively updating a latent state has emerged as a powerful paradigm for reasoning. Yet the internal mechanisms that enable these iterative models to generalize beyond memorized patterns remain unclear. We hypothesize that generalizable reasoning arises from learning task-conditioned attractors: latent dynamical systems whose stable fixed points correspond to valid solutions. We formalize this process through Equilibrium Reasoners (EqR), which enable test-time scaling without external verifiers or task-specific priors. EqR scales internal dynamics along two axes: depth, by running more iterations, and breadth, by aggregating stochastic trajectories from multiple initializations. Empirically, gains from test-time scaling are tightly coupled with stronger convergence toward solution-aligned attractors. This attractor perspective allows neural networks to adaptively allocate test-time compute based on task difficulty. While simple cases converge within 1 to 5 iteration steps, harder cases benefit from massive test-time scaling. By unrolling up to the equivalent of 40,000 layers, scalable latent reasoning boosts accuracy from 2.6% for feedforward models to over 99% on Sudoku-Extreme. These results suggest that learned attractor landscapes provide a useful mechanistic lens for understanding scalable reasoning in iterative latent models.

可扩展推理吸引子测试时扩展

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。