用数学范畴论统一描述深度学习模型结构。
Weaves, Wires, and Morphisms: Formalizing and Implementing the Algebra of Deep Learning
- 提出轴步与数组广播的范畴论形式化方法
- 支持模型组件的代数组合与精确函数表达
- 提供跨语言实现，适合模型设计研究者
尽管深度学习模型运行在明确的数学函数上，但我们缺乏描述模型架构的正式数学框架。现有符号、图示和伪代码难以处理非线性广播及组件与整体模型的关系。本文引入范畴论框架，通过新提出的轴步类别和数组广播类别，形式化广播机制。该框架可精确表达并组合模型底层函数。这些数学定义被转化为人类可读图示与机器可处理数据结构。我们提供了Python（pyncd）和TypeScript（tsncd）的镜像实现，支持代数构建、图转换、PyTorch编译与图渲染。该框架为深度学习模型的设计与分析奠定了系统化、形式化的基础。
原文摘要 · Abstract (English)
Despite deep learning models running well-defined mathematical functions, we lack a formal mathematical framework for describing model architectures. Ad-hoc notation, diagrams, and pseudocode poorly handle nonlinear broadcasting and the relationship between individual components and composed models. This paper introduces a categorical framework for deep learning models that formalizes broadcasting through the novel axis-stride and array-broadcasted categories. This allows the mathematical function underlying architectures to be precisely expressed and manipulated in a compositional manner. These mathematical definitions are translated into human manageable diagrams and machine manageable data structures. We provide a mirrored implementation in Python (pyncd) and TypeScript (tsncd) to show the universal aspect of our framework, along with features including algebraic construction, graph conversion, PyTorch compilation and diagram rendering. This lays the foundation for a systematic, formal approach to deep learning model design and analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。