用向量符号架构构建可证明多项式终止的编程语言,让神经网络学会类型。
Hey Pentti, We Did It Again!: Differentiable vector-symbolic types that prove polynomial termination
- 用全同构的向量符号编码程序与类型,保证所有程序在多项式时间内终止。
- 类型在嵌入空间中邻近点具有相似结构和内容,支持神经网络学习类型。
- 为模拟人类技能习得速度提供新范式,比现有方法更高效。
我们提出一种类型化编程语言 Doug,其中所有类型程序均可在多项式时间内证明终止,其编码基于向量符号架构(VSA)。Doug 是对轻线性函数式编程语言(LLFPL)的实现,其类型采用全息声明性记忆(HDM)的槽-值编码方案,术语则使用 Flanagan(2024)提出的 Lisp VSA 变体。该系统使神经网络嵌入空间中的某些点可被解释为类型,且邻近点在结构和内容上相似,从而支持类型的学习。根据 Chollet(2019)、Card(1983)和 Newell(1981)的观点,技能即满足目标的动作程序,技能习得可视为程序合成。我们希望利用 Doug 揭示一种接近人类节奏的技能学习机制(远快于暴力搜索;Heathcote, 2000),超越当前所有方法(Kaplan, 2020; Jones, 2021; Chollet, 2024)的效率。该方法推动了对真实脑内表征及其学习过程的建模。
原文摘要 · Abstract (English)
We present a typed computer language, Doug, in which all typed programs may be proved to halt in polynomial time, encoded in a vector-symbolic architecture (VSA). Doug is just an encoding of the light linear functional programming language (LLFPL) described by (Schimanski2009, ch. 7). The types of Doug are encoded using a slot-value encoding scheme based on holographic declarative memory (HDM; Kelly, 2020). The terms of Doug are encoded using a variant of the Lisp VSA defined by (Flanagan, 2024). Doug allows for some points on the embedding space of a neural network to be interpreted as types, where the types of nearby points are similar both in structure and content. Types in Doug are therefore learnable by a neural network. Following (Chollet, 2019), (Card, 1983), and (Newell, 1981), we view skill as the application of a procedure, or program of action, that causes a goal to be satisfied. Skill acquisition may therefore be expressed as program synthesis. Using Doug, we hope to describe a form of learning of skilled behaviour that follows a human-like pace of skill acquisition (i.e., substantially faster than brute force; Heathcote, 2000), exceeding the efficiency of all currently existing approaches (Kaplan, 2020; Jones, 2021; Chollet, 2024). Our approach brings us one step closer to modeling human mental representations, as they must actually exist in the brain, and those representations' acquisition, as they are actually learned.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。