用图示方法加速群等变神经网络的计算效率
A Diagrammatic Approach to Improve Computational Efficiency in Group Equivariant Neural Networks
- 基于范畴论构建图示框架，将权重矩阵表示为图形组合
- 对四类群实现指数级提速，时间复杂度显著降低
- 适合研究对称性建模与高效神经网络设计的学者
群等变神经网络因其在具有已知对称性的数据上具备良好泛化能力而日益重要。近期研究表明，采用高阶张量幂空间作为层结构的这类网络具有巨大潜力，但其实际应用受限于计算成本过高。本文提出一种针对对称群、正交群、特殊正交群和辛群的快速矩阵乘法算法，适用于映射张量幂层空间的等变权重矩阵。该算法基于范畴论构建的图示框架，可将每个权重矩阵表示为一组图形的线性组合，并利用这些图形将原始计算分解为一系列最优步骤。实验表明，相比朴素矩阵乘法，该算法在时间复杂度上实现了指数级优化。
原文摘要 · Abstract (English)
Group equivariant neural networks are growing in importance owing to their ability to generalise well in applications where the data has known underlying symmetries. Recent characterisations of a class of these networks that use high-order tensor power spaces as their layers suggest that they have significant potential; however, their implementation remains challenging owing to the prohibitively expensive nature of the computations that are involved. In this work, we present a fast matrix multiplication algorithm for any equivariant weight matrix that maps between tensor power layer spaces in these networks for four groups: the symmetric, orthogonal, special orthogonal, and symplectic groups. We obtain this algorithm by developing a diagrammatic framework based on category theory that enables us to not only express each weight matrix as a linear combination of diagrams but also makes it possible for us to use these diagrams to factor the original computation into a series of steps that are optimal. We show that this algorithm improves the Big-$O$ time complexity exponentially in comparison to a naïve matrix multiplication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。