发现超位置推理的涌现机制：莫比乌斯吸引子与级联监督协同作用。
Emergence of Frontier Superposition: Möbius attractor and Cascade Supervision
- 通过莫比乌斯吸引子和级联监督，实现深度并行推理
- 实验验证级联监督使最终余弦值达0.69，优于传统方法的0.37
- 适合研究大模型推理机制与优化设计的研究者
超位置使Transformer能够并行承载整个推理前沿，在有限深度前向传播中替代串行思维链。尽管Zhu等人（2025）手工构造了在单个残差流中的等权重广度优先前沿以解决图可达性问题，但梯度下降是否能在置换对称鞍点中找到该目标仍未知。本文在Erdős-Rényi图上解决了这一问题，分离了架构与监督的贡献。架构上，识别出莫比乌斯吸引子：在树状情形下，$S_n$-对称性使逐层动态简化为一维莫比乌斯映射，其零集为包含等权重超位置态的全维全局最优超平面。监督上，提出级联监督：一种损失类，其反向传播同时实现（A）选择性引导、（B）跨深度梯度持续、（C）每步判别能力（如$\/mathcal{L}_{sup}$与$\/mathcal{L}_{node}$）。端到端监督无法满足条件（B），内部梯度在第c层衰减为$(np)^{-(D-c-2)/2}$，在抵达超平面前停滞。结论：莫比乌斯吸引子 + 级联监督 = 超位置推理的涌现。无参数衰减定律预测深度D=3时，最终余弦值分别为0.35（端到端）与0.71（级联），实验结果为0.37与0.69，每步误差小于0.02。
原文摘要 · Abstract (English)
Superposition allows Transformers to reason in depth, carrying an entire reasoning frontier in parallel through a bounded-depth forward pass instead of unrolling serial chain-of-thought tokens. While Zhu et al. (2025) hand-crafted an equal-weight breadth-first frontier in a single residual stream for graph reachability, it remained open whether gradient descent could ever find this target amidst permutation-symmetric saddles. We close this gap on Reachability-by-Superposition over Erdős-Rényi graphs by isolating architectural and supervisional contributions. Architecturally, we identify a Möbius attractor: under $S_n$-symmetry in the tree regime, layerwise dynamics reduce to a 1D Möbius map whose zero set is a codimension-one manifold of global optima containing the equal-weight superposition state. On the supervision side, we identify Cascade Supervision: a loss class whose backward pass simultaneously delivers (A) selectivity bootstrap, (B) gradient persistence across depth, and (C) per-step discrimination (e.g., \mathcal{L}_{sup} and \mathcal{L}_{node}). End-to-end supervision fails condition (B) and is provably insufficient: internal gradients at layer c decay as (np)^{-(D-c-2)/2} in the graph fan-out and stall before the manifold is reached. Our thesis: Möbius attractor + Cascade Supervision = emergence of superposition reasoning. The parameter-free decay law predicts a final-step cosine of 0.35 vs. 0.71 (end-to-end vs. cascade) at depth D=3; experiments confirm 0.37 vs. 0.69, matching within 0.02 at every step.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。