arXiv:2603.23577cs.LGcs.CL2026-03被引 1

LLM逻辑推理需拓扑变形,非线性动态构建离散决策边界。

The Geometric Price of Discrete Logic: Context-driven Manifold Dynamics of Number Representations

  • 通过格拉姆-施密特分解揭示激活向量的双模调控机制。
  • 消除代数发散分量后,奇偶分类准确率从100%降至38.57%。
  • 适用于理解模型推理失效、幻觉与讨好行为的几何成因。

大型语言模型(LLMs)在连续语义空间中泛化良好,但严格逻辑推理需要形成离散决策边界。现有依赖线性保距投影的理论无法解决这一根本矛盾。本文提出,任务上下文作为非保距的动力学算子,强制产生必要的‘拓扑扭曲’。通过对残差流激活应用格拉姆-施密特分解,我们发现一种双重调控机制:一类无关类别的拓扑保持,锚定全局结构以防止语义坍缩;另一类特定代数发散,定向撕裂跨类别概念以形成逻辑边界。我们在从简单映射到复杂素性判断的任务梯度上验证了这种几何演化。关键地,针对性向量消融证明了该拓扑与模型功能的严格因果关联:消除代数发散分量使奇偶分类准确率从100%降至随机水平(38.57%)。此外,我们揭示了分层的三阶段几何动态,并发现社交压力提示下模型无法生成足够发散,导致‘流形纠缠’,从而几何解释了讨好与幻觉现象。最终,我们的发现修正了线性保距假设,表明LLMs中离散逻辑的出现需以不可还原的拓扑变形为代价。

原文摘要 · Abstract (English)

Large language models (LLMs) generalize smoothly across continuous semantic spaces, yet strict logical reasoning demands the formation of discrete decision boundaries. Prevailing theories relying on linear isometric projections fail to resolve this fundamental tension. In this work, we argue that task context operates as a non-isometric dynamical operator that enforces a necessary "topological distortion." By applying Gram-Schmidt decomposition to residual-stream activations , we reveal a dual-modulation mechanism driving this process: a class-agnostic topological preservation that anchors global structure to prevent semantic collapse, and a specific algebraic divergence that directionally tears apart cross-class concepts to forge logical boundaries. We validate this geometric evolution across a gradient of tasks, from simple mapping to complex primality testing. Crucially, targeted specific vector ablation establishes a strict causal binding between this topology and model function: algebraically erasing the divergence component collapses parity classification accuracy from 100% to chance levels (38.57%). Furthermore, we uncover a three-phase layer-wise geometric dynamic and demonstrate that under social pressure prompts, models fail to generate sufficient divergence. This results in a "manifold entanglement" that geometrically explains sycophancy and hallucination. Ultimately, our findings revise the linear-isometric presumption, demonstrating that the emergence of discrete logic in LLMs is purchased at an irreducible cost of topological deformation.

逻辑推理拓扑变形流形分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。