用新架构实现更小、更准、可验证的自适应AI,适合领域专用模型
Adaptive Domain Models: Bayesian Evolution, Warm Rotation, and Principled Training for Geometric and Neuromorphic AI
- 基于几何代数与正数算术,实现内存恒定、梯度精确的训练
- 训练内存仅约为推理占用的两倍,支持神经形态与通用模型统一训练
- 提出贝叶斯蒸馏与热旋转机制,解决数据少和部署中断问题
现有AI训练依赖反向自动微分与IEEE-754浮点运算,导致训练内存开销大、优化器复杂且几何结构退化。本文基于三项前期成果:维度类型系统与确定性内存管理(Haynes 2026),实现栈兼容梯度分配与精确quire累积;程序超图(Haynes 2026),确保几何代数计算中阶数不变性;b-posit有界设计(Jonnalagadda et al. 2025),使正数算术在传统推理硬件上可行。三者结合实现深度无关的训练内存上限约等于推理占用的两倍,保持权重更新的阶数不变性,并支持精确梯度累积,适用于损失函数优化与脉冲时序依赖的神经形态模型。提出贝叶斯蒸馏,从通用模型中提取潜在先验结构,解决领域专用训练的数据稀缺问题。部署阶段引入热旋转机制,模型更新可无缝过渡至推理路径,正确性通过PHG证书与签名版本记录保障。最终系统为更小、更精确、持续自适应、物理结构可验证的领域专用AI,可由现有模型初始化。
原文摘要 · Abstract (English)
Prevailing AI training assumes reverse-mode automatic differentiation over IEEE-754 arithmetic. The memory overhead of training relative to inference, optimizer complexity, and structural degradation of geometric properties through training are consequences of this arithmetic substrate. This paper develops an alternative training architecture grounded in three prior results: the Dimensional Type System and Deterministic Memory Management framework (Haynes 2026), which establishes stack-eligible gradient allocation and exact quire accumulation as design-time verifiable properties; the Program Hypergraph (Haynes 2026), which establishes grade preservation through geometric algebra computations as a type-level invariant; and the b-posit bounded-regime design (Jonnalagadda et al. 2025), which makes posit arithmetic tractable across hardware targets conventionally considered inference-only. Their composition enables depth-independent training memory bounded to approximately twice the inference footprint, grade-preserving weight updates, and exact gradient accumulation, applicable uniformly to loss-function-optimized and spike-timing-dependent neuromorphic models. We introduce *Bayesian distillation*, a mechanism by which the latent prior structure of a general-purpose model is extracted through the ADM training regime, resolving the data-scarcity bootstrapping problem for domain-specific training. For deployment, we introduce *warm rotation*, an operational pattern in which an updated model transitions into an active inference pathway without service interruption, with correctness formalized through PHG certificates and signed version records. The result is a class of domain-specific AI systems that are smaller and more precise than general-purpose models, continuously adaptive, verifiably correct with respect to the physical structure of their domains, and initializable from existing models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。