通过谱特性提升高阶神经网络表达能力，有效缓解参数爆炸问题。
Spectral Higher-Order Neural Networks Have Sharp Expressivity Bounds
- 利用谱属性设计参数共享机制，减少高阶神经网络参数量。
- 在N位异或任务上表现优异，验证了其强大的表达能力。
- 适合需要高效高表达模型的场景，如复杂函数逼近。
神经超图是现代机器学习中神经网络的自然推广。然而，其部署面临挑战：所需加权超边数量导致参数急剧膨胀。近期提出的基于谱特性的新参数化方法，通过权重共享机制复用参数，显著降低了计算成本。初步测试表明，谱高阶架构在性能和可解释性方面均有提升。本文进一步在著名的N位异或任务上评估该框架，该任务以极高的挑战性著称。我们论证，谱高阶神经网络（SHONNs）具备高度灵活且可调的假设空间。
原文摘要 · Abstract (English)
Neural hypergraphs are a natural generalization of neural networks, the reference models in modern machine learning. Yet, their deployment has proven demanding: the number of weighted hyperedges required leads to an intractable parameter explosion. However, a novel parametrization that leverages spectral attributes for neural hypergraphs has been recently proposed, that enables to recycle parameters via a weight sharing scheme and consequently yields a significant reduction of the associated computational cost. Preliminary tests carried out on spectral higher-order architectures pointed to meaningful improvements in both performance and interpretability. Building on these results, we advance the benchmarking efforts by evaluating the spectral higher order framework on N-bit parity tasks, a well-established testbed known to be particularly challenging. As we will convincingly argue, Spectral Higher-Order Neural Networks (SHONNs) possess a versatile and highly tunable hypothesis space.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。