用范畴论重构机器学习,从微小扰动中捕捉非组合性缺陷。
Learning in Infinitesimal Non-Compositional Sketches
- 提出LINCS框架,通过切范畴分析学习模型的微小扰动是否破坏组合性。
- 定义了增量切变算子,将模型迭代提升至更高阶切空间,形成因子化塔。
- 适用于深度学习、大模型与强化学习,为模型鲁棒性提供新理论视角。
本文提出一种范畴论框架——极小非组合性草图学习(LINCS),用于修复非组合性:即图表无法通过商草图提升到切范畴设置下的泛性质因子化问题。机器学习问题被形式化为草图:带有交换条件 $\\(mathcal D$,极限锥 $\\(mathcal L$,余极限共锥 $\\(mathcal K$ 的图,推广了传统损失函数标量化解或向量空间假设。非组合性被定义为泛因子化问题的失败,而非预测值间的算术误差。给定学习草图 $\\(\mathbb S=(S,\\(mathcal D,\\(mathcal L,\\(mathcal K)$,其基础图为 $S$,以及模型 $D:J ightarrow C$,基础缺陷是因子化 $\\(\mbox{Obs}(\\(\mbox{Fact}_{\\(\mathbb S}(D))$ 的障碍。切提升将切函子 $T$ 应用于 $D$ 得到 $TD:J ightarrow C$,LINCS 定义为障碍 $\\(\mbox{Obs}(\\(\mbox{Fact}_{\\(\mathbb S}(TD))$ —— 即判断无穷小扰动是否保持组合性约束。本文还引入切学习草图,即带有 Cockett-Cruttwell 切结构的草图。定义了 INC 自函子,迭代切提升,生成因子化问题塔 $D, TD, T^2D, \cdots$。机器学习因此被建模为寻找一个余代数不动点,使得连续切展开趋于稳定($νT_{\\(\mbox{INC}}$)。利用 Aczel--Mendler 定理,证明当 $T_{\\(\mbox{INC}}$ 具有基于集合的类实现并生成最终载体时,最终 INC 余代数存在。对 LINCS 的详细实验评估正在进行,涵盖深度学习、大语言模型和强化学习等多个具体场景,详见配套论文。
原文摘要 · Abstract (English)
This paper develops a categorical framework -- Learning in Infinitesimal Non-Compositional Sketches (LINCS) -- as the repair of non-compositionality: failures of diagrams to factor through quotient sketches lifted to the tangent category setting. Machine learning problems are specified as sketches: graphs with commutativity conditions $\mathcal D$, limit cones $\mathcal L$, and colimit cocones $\mathcal K$, generalizing the usual scalarization of loss functions or vector space assumptions. Non-compositionality is defined purely as failure of a universal factorization problem, not as arithmetic error between the desired and actual predictions. Given a learning sketch $\mathbb S=(S,\mathcal D,\mathcal L,\mathcal K)$, whose underlying graph is $S$, and a model $D:J \rightarrow C$, the base defect is the obstruction to factorization $\mbox{Obs}(\mbox{Fact}_{\mathbb S}(D))$. The tangent lift applies the tangent functor $T$ to obtain $TD:J \rightarrow C$, and LINCS is defined as the obstruction $\mbox{Obs}(\mbox{Fact}_{\mathbb S}(TD))$ -- asking whether infinitesimal perturbations preserve the compositionality constraints.The paper also introduces Tangent Learning Sketches, which are sketches equipped with Cockett-Cruttwell tangent structure. The paper defines the INC endofunctor, which iterates the tangent lift, producing a tower $D,TD,T^2D, \cdots$ of factorization problems. ML is thereby formulated as the search for a coalgebraic fixed point where successive tangent unfoldings stabilize ($νT_{\mbox{INC}}$). Using the Aczel--Mendler theorem, we prove existence of a final INC coalgebra whenever $T_{\mbox{INC}}$ admits a set-based class realization that creates its final carrier. A detailed experimental evaluation of LINCS is underway in a number of concrete ML settings, including deep learning, large language models, and reinforcement learning, and is described in companion papers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。