用几何新框架解决大模型幻觉和遗忘问题。
Statistically Meaningful Geometry (SMG) Beyond the Euclidean Paradigm, with Application to Generative AI
- 将模型提升到无限维统计流形,用纤维丛建模优化空间。
- 过滤内部噪声后,生成结果方差被几何边界约束。
- 适合研究模型可靠性与生成安全性的学者。
传统一致收敛界与经验风险最小化在大规模过参数化模型(如大语言模型和生物序列网络)中失效。由于近乎无限的自由度,其优化景观形成平坦的垂直规范谷,使经典泛化度量无效,并引发严重病理现象,如生成幻觉与灾难性遗忘。我们提出统计意义几何(SMG)框架,将确定性参数模型升维至无限维非参数Orlicz统计流形。将全状态空间建模为微分流丛($/mathcal{M}, /mathcal{B}, π, /mathcal{V}, /mathcal{H}, ω$),建立双重推断范式。将Ehresmann联络1-形式$ω$定义为动态几何滤波器,剥离垂直规范噪声(结构内部方向,SID),分离出严格非退化的水平分布(统计变分方向,SVD$χ$)。证明在联络滤波预训练下,分布外预测方差严格受可识别商基流形$/mathcal{B}$有限直径上界控制,确立生成幻觉的硬几何约束。通过将下游更新投影至历史水平载体的正交补空间,形式化SMG顺序适应流,证明灾难性遗忘可完全非渐近消除。SMG以坐标无关拓扑约束取代经验微调启发式,连接先进微分几何与人工智能结构可靠性。
原文摘要 · Abstract (English)
Conventional uniform convergence bounds and empirical risk minimization break down in massive over-parameterized models, such as large language transformers and biological sequence networks. With near-infinite unconstrained internal degrees of freedom, their optimization landscapes develop flat vertical gauge valleys, rendering classical generalization metrics vacuous and inducing severe pathologies, specifically generative hallucination and catastrophic forgetting. We introduce the Statistically Meaningful Geometry (SMG) framework, an information-geometric paradigm lifting deterministic parametric models into infinite-dimensional non-parametric Orlicz statistical manifolds. Modeling the total state space as a differential fiber bundle ($\mathcal{M}, \mathcal{B}, π, \mathcal{V}, \mathcal{H}, ω$), we establish a Two-Fold Inference Paradigm. We formalize an Ehresmann connection 1-form $ω$ as a dynamic geometric filter that strips away vertical gauge noise (Structural Internal Directions, or SID) and isolates learning trajectories along the strictly non-degenerate horizontal distribution (Statistical Variational Directions, or SVD$χ$). We prove that under connection-filtered pre-training, out-of-distribution predictive variance is strictly upper-bounded by the finite diameter of the identifiable quotient base manifold $\mathcal{B}$, establishing a hard geometric containment of generative hallucinations. By projecting downstream updates onto the orthogonal complement of the historical horizontal carriage, we formalize the SMG Sequential Adaptation Flow, proving the total non-asymptotic elimination of catastrophic forgetting. SMG replaces empirical fine-tuning heuristics with coordinate-free topological constraints, bridging advanced differential geometry with structural reliability in AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。