arXiv:2601.18702cs.LGcs.AI2026-01被引 1

用精确算术取代浮点数,让大模型推理更稳定高效

From Fuzzy to Exact: The Halo Architecture for Infinite-Depth Reasoning via Rational Arithmetic

  • 采用有理数域和双环结构,解决精确计算的位宽膨胀问题
  • 移除梯度裁剪等数值保护机制,实现训练收敛时间数量级下降
  • 适合追求数学严谨性与极致推理效率的研究者

当前大型语言模型的扩展范式依赖于'模糊'的浮点数运算。为缓解这种近似基础带来的不稳定性,现有架构引入了复杂数值与结构启发式方法——如复杂残差、预-RMSNorm、注意力缩放和梯度裁剪——消耗大量算力以防止数值崩溃。本文提出向'精确'范式的转变,引入基于有理数域(Q)的Halo架构,并配备定制的精确推理单元(EIU)。为应对有理数运算中指数级增长的位宽问题,Halo采用双环拓扑:微环(连续性维持)通过丢番图逼近严格控制内存复杂度;宏环(符号对齐)则通过周期性状态坍缩保证逻辑一致性。该稳定双环基底使'大规模拆除'数值保护机制成为可能,将Transformer模块还原为纯粹代数形式(Tabula Rasa)。此外,我们验证了'效率悖论':消除梯度噪声(sigma → 0)后可使用极大学习率,有望将总收敛时间减少至原时间的数量级。Halo证明,通用智能需要在严格数学框架下融合连续域与离散链。

原文摘要 · Abstract (English)

The prevailing scaling paradigm of Large Language Models (LLMs) rests on a substrate of "Fuzzy" floating-point arithmetic. To mitigate the inherent instability of this approximate foundation, modern architectures have erected a complex scaffolding of structural and numerical heuristics--Complex Residuals, Pre-RMSNorm, Attention Scaling, and Gradient Clipping--consuming significant compute solely to prevent numerical collapse. We propose a paradigm shift to the "Exact". We introduce the Halo Architecture, grounded in the Rational Field (Q) and powered by a custom Exact Inference Unit (EIU). To resolve the exponential bit-width growth of rational arithmetic, Halo employs a Dual-Ring Topology that unifies two complementary control mechanisms: (1) The Micro-Ring (Continuum Maintenance), which strictly bounds memory complexity via Diophantine Approximation; and (2) The Macro-Ring (Symbolic Alignment), which enforces logical consistency via periodic state collapse. This stable dual-ring substrate allows for the "Great Dismantling" of numerical scaffolding, reducing the Transformer block to its "Clean" algebraic form (Tabula Rasa). Furthermore, we verify the "Efficiency Paradox": the elimination of gradient noise (sigma -> 0) allows for Macro-Learning Rates, potentially reducing the Total Time-to-Convergence by orders of magnitude. Halo demonstrates that General Intelligence requires the hybridization of continuous fields and discrete chains under a rigorous mathematical framework.

大模型精确计算推理架构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。