用代数乘积统一建模神经网络层，揭示结构本质
Product Interaction: An Algebraic Formalism for Deep Learning Architectures
- 将网络层构建为代数乘积的组合，按阶次递增组织表达式
- 发现卷积、注意力等架构均属不同阶次的乘积交互形式
- 适合研究模型结构原理或设计新网络的研究者
本文提出乘积交互（product interactions），一种基于特定代数上乘法算子的代数形式化方法，用于构建神经网络层。该形式化通过增加交互阶次，系统地生成和组织代数表达式。核心观察是：现代神经网络中的代数表达式可统一归为线性、二次及高阶乘积交互。卷积与等变网络是受对称性约束的线性乘积交互，而注意力机制与Mamba则对应更高阶的乘积交互。
原文摘要 · Abstract (English)
In this paper, we introduce product interactions, an algebraic formalism in which neural network layers are constructed from compositions of a multiplication operator defined over suitable algebras. Product interactions provide a principled way to generate and organize algebraic expressions by increasing interaction order. Our central observation is that algebraic expressions in modern neural networks admit a unified construction in terms of linear, quadratic, and higher-order product interactions. Convolutional and equivariant networks arise as symmetry-constrained linear product interactions, while attention and Mamba correspond to higher-order product interactions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。