用范畴论统一向量符号架构,让模型既可解释又兼容神经网络。
Developing a Foundation of Vector Symbolic Architectures Using Category Theory
- 将向量符号架构升格为上预层,用范畴论重新定义其运算
- 证明右肯扩展可化为逐元素操作,计算更高效
- 为可解释性模型设计提供新思路,适合研究符号与神经融合
连接主义机器学习方法(如神经网络)当前广受欢迎,但需大量数据且模型难以解释。向量符号架构(VSAs)是一种兼容神经网络与梯度学习的替代框架,能显式建模组合性,源自认知科学中统一神经处理与人类符号推理的需求。尽管机器学习已受益于范畴论分析,VSAs尚无类似研究。本文首次尝试将范畴论应用于VSAs:将向量推广至上预层,将VSA运算描述为外张量积的右肯扩展。我们证明在这些情形下,右肯扩展可表达为简单的逐元素操作。通过具体例子验证该形式化,关联现有实现,并提出新的可能设计。
原文摘要 · Abstract (English)
Connectionist approaches to machine learning, \emph{i.e.} neural networks, are enjoying a considerable vogue right now. However, these methods require large volumes of data and produce models that are uninterpretable to humans. An alternative framework that is compatible with neural networks and gradient-based learning, but explicitly models compositionality, is Vector Symbolic Architectures (VSAs). VSAs are a family of algebras on high-dimensional vector representations. They arose in cognitive science from the need to unify neural processing and the kind of symbolic reasoning that humans perform. While machine learning methods have benefited from category-theoretical analyses, VSAs have not yet received similar treatment. In this paper, we present a first attempt at applying category theory to VSAs. Specifically, We generalise from vectors to co-presheaves, and describe VSA operations as the right Kan extensions of the external tensor product. This formalisation involves a proof that the right Kan extension in such cases can be expressed as simple, element-wise operations. We validate our formalisation with worked examples that connect to current VSA implementations, while suggesting new possible designs for VSAs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。