大模型推理的几何结构随规模变化，不同领域呈现不同演化模式。
The Geometry of Thought: How Scale Restructures Reasoning In Large Language Models
- 通过分析万级推理轨迹，发现规模引发领域特异的相变。
- 法律推理维度下降45%，代码推理形成离散策略格子，科学数学保持几何不变。
- 提出神经推理算子，可预测推理终点，加速推理过程。
规模并未均匀提升推理能力，而是重构其结构。分析跨四个领域（法律、科学、代码、数学）和两种规模（80亿、700亿参数）的25,000+条思维链轨迹，发现神经缩放定律引发领域特异的相变：法律推理经历‘结晶化’——表示维度下降45%（d95: 501 → 274），轨迹对齐提升31%，流形解缠10倍；科学与数学推理保持‘液态’，几何不变性不受9倍参数增长影响；代码推理形成离散的‘晶格’策略模式（轮廓度0.13 → 0.42）。该几何结构可预测可学习性。我们引入神经推理算子，实现从初始到终态隐藏状态的映射，在结晶型法律推理中，仅用探测解码即达63.6%准确率，无需遍历中间步骤。进一步识别出跨领域与规模的通用振荡特征（相干性约-0.4），表明注意力与前馈层通过相反动力驱动推理。研究揭示思考成本由流形几何决定而非任务难度，为拓扑允许的推理加速提供蓝图。
原文摘要 · Abstract (English)
Scale does not uniformly improve reasoning - it restructures it. Analyzing 25,000+ chain-of-thought trajectories across four domains (Law, Science, Code, Math) and two scales (8B, 70B parameters), we discover that neural scaling laws trigger domain-specific phase transitions rather than uniform capability gains. Legal reasoning undergoes Crystallization: 45% collapse in representational dimensionality (d95: 501 -> 274), 31% increase in trajectory alignment, and 10x manifold untangling. Scientific and mathematical reasoning remain Liquid - geometrically invariant despite 9x parameter increase. Code reasoning forms a discrete Lattice of strategic modes (silhouette: 0.13 -> 0.42). This geometry predicts learnability. We introduce Neural Reasoning Operators - learned mappings from initial to terminal hidden states. In crystalline legal reasoning, our operator achieves 63.6% accuracy on held-out tasks via probe decoding, predicting reasoning endpoints without traversing intermediate states. We further identify a universal oscillatory signature (coherence ~ -0.4) invariant across domains and scales, suggesting attention and feedforward layers drive reasoning through opposing dynamics. These findings establish that the cost of thought is determined not by task difficulty but by manifold geometry - offering a blueprint for inference acceleration where topology permits.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。