arXiv:2504.08075cs.LOcs.LG2025-04被引 1

用数学几何揭示算法本质,发现不同实现的程序在贝叶斯推断中可被区分

Programs as Singularities

  • 将图灵机编码嵌入光滑参数空间,以势函数临界点表示程序
  • 程序内部结构由势函数泰勒展开中的错误综合征决定
  • 揭示贝叶斯后验能区分同功能但不同实现的算法

我们建立图灵机结构与实解析函数奇点结构之间的对应关系,通过将线性逻辑中的Ehrhard-Regnier导数与森本正夫奇异学习理论中的几何角色相联系。方法是将普通离散图灵机代码嵌入一组噪声编码构成的光滑参数空间,在该空间上定义一个以图灵机为临界点的势函数。通过将该势函数在临界点的泰勒展开与错误综合征的组合结构关联,建立了局部几何与图灵机内部结构之间的联系。该势函数为某统计模型的负对数似然,因此图灵机结构及其相关奇点与贝叶斯推断紧密相关。两个产生相同预测函数的算法可能对应具有不同几何特征的奇点,表明贝叶斯后验能够区分不同的算法实现,这与纯粹的功能主义推理观点相悖。在奇异学习理论背景下,本研究指向对奥卡姆剃刀及归纳推理中‘简洁性’含义的更精细理解。

原文摘要 · Abstract (English)

We develop a correspondence between the structure of Turing machines and the structure of singularities of real analytic functions, based on connecting the Ehrhard-Regnier derivative from linear logic with the role of geometry in Watanabe's singular learning theory. The correspondence works by embedding ordinary (discrete) Turing machine codes into a family of noisy codes which form a smooth parameter space. On this parameter space we consider a potential function which has Turing machines as critical points. By relating the Taylor series expansion of this potential at such a critical point to combinatorics of error syndromes, we relate the local geometry to internal structure of the Turing machine. The potential in question is the negative log-likelihood for a statistical model, so that the structure of the Turing machine and its associated singularity is further related to Bayesian inference. Two algorithms that produce the same predictive function can nonetheless correspond to singularities with different geometries, which implies that the Bayesian posterior can discriminate between distinct algorithmic implementations, contrary to a purely functional view of inference. In the context of singular learning theory our results point to a more nuanced understanding of Occam's razor and the meaning of simplicity in inductive inference.

图灵机贝叶斯推断奇异学习几何推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。