用范畴论为深度学习提供理论根基,统一模型设计与分析框架。
Towards a Categorical Foundation of Deep Learning: A Survey
- 以范畴论视角重构深度学习核心机制,如梯度学习与网络结构。
- 通过函子和弦图等工具保持抽象层次间的结构一致性。
- 适合对理论建模、机器学习形式化感兴趣的科研人员参考。
机器学习研究进展迅猛,但缺乏坚实的理论基础,许多关键成果源于难以解释的直觉设计,导致研究债务累积且可复现性差。本文是一篇综述,聚焦于近期将范畴论应用于机器学习的研究。范畴论作为抽象数学的分支,在多个领域已有成功应用,可作为数学与科学的通用语言,有望为机器学习提供统一的结构框架,解决当前面临的诸多问题。本文重点探讨范畴论在深度学习中的应用:利用范畴光学建模基于梯度的学习过程;通过范畴代数与积分变换连接经典计算机科学与神经网络;使用函子在不同抽象层次间传递结构信息;以及借助弦图实现神经网络架构的精细可视化表示。
原文摘要 · Abstract (English)
The unprecedented pace of machine learning research has lead to incredible advances, but also poses hard challenges. At present, the field lacks strong theoretical underpinnings, and many important achievements stem from ad hoc design choices which are hard to justify in principle and whose effectiveness often goes unexplained. Research debt is increasing and many papers are found not to be reproducible. This thesis is a survey that covers some recent work attempting to study machine learning categorically. Category theory is a branch of abstract mathematics that has found successful applications in many fields, both inside and outside mathematics. Acting as a lingua franca of mathematics and science, category theory might be able to give a unifying structure to the field of machine learning. This could solve some of the aforementioned problems. In this work, we mainly focus on the application of category theory to deep learning. Namely, we discuss the use of categorical optics to model gradient-based learning, the use of categorical algebras and integral transforms to link classical computer science to neural networks, the use of functors to link different layers of abstraction and preserve structure, and, finally, the use of string diagrams to provide detailed representations of neural network architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。