提出新型神经网络架构，实现通用前馈网络中的高阶交互
Spectral Higher-Order Neural Networks
- 基于谱属性重构模型，支持任意前馈网络的高阶交互
- 解决高阶权重传播带来的稳定性与参数膨胀问题
- 适合需要建模复杂非线性关系的通用深度学习任务
神经网络是现代机器学习的核心工具。传统架构假设单元间仅存在逐层前向传播的二元交互。已有研究尝试拓展至更高阶耦合，但通常仅作为图神经网络的增强形式，且仅在输入具有显式超图结构时才具优势。本文提出谱高阶神经网络（SHONNs），一种通用前馈网络中引入高阶交互的新方法。该方法通过谱属性重述模型，有效缓解加权高阶前向传播带来的稳定性问题与参数规模增长难题。
原文摘要 · Abstract (English)
Neural networks are fundamental tools of modern machine learning. The standard paradigm assumes binary interactions (across feedforward linear passes) between inter-tangled units, organized in sequential layers. Generalized architectures have been also designed that move beyond pairwise interactions, so as to account for higher-order couplings among computing neurons. Higher-order networks are however usually deployed as augmented graph neural networks (GNNs), and, as such, prove solely advantageous in contexts where the input exhibits an explicit hypergraph structure. Here, we present Spectral Higher-Order Neural Networks (SHONNs), a new algorithmic strategy to incorporate higher-order interactions in general-purpose, feedforward, network structures. SHONNs leverages a reformulation of the model in terms of spectral attributes. This allows to mitigate the common stability and parameter scaling problems that come along weighted, higher-order, forward propagations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。