用张量分解替代符号回归，提升任务驱动神经元的稳定性和效率
NeuronSeek: On Stability and Expressivity of Task-driven Neurons
- 用张量分解发现最优神经元形式，替代传统符号回归
- 理论证明固定参数量网络可任意逼近连续函数
- 在多个基准上表现优于现有模型，兼具稳定与高效
受人脑为不同任务设计特定神经元的启发，近年来深度学习探索了通过修改网络神经元来构建任务驱动神经元。本文提出的 NeuronSeek 框架采用符号回归（SR）发现最优神经元表达式并构建网络。本工作将其替换为张量分解（TD），显著提升了稳定性并加快收敛速度。此外，我们建立了理论保证：通过常见激活函数修改聚合函数，固定参数量的网络可任意逼近任意连续函数，为 NeuronSeek 框架提供了严格的数学基础。大量实验证明，NeuronSeek-TD 在多个基准上不仅稳定性更优，且性能媲美当前最先进模型。代码已开源：https://github.com/HanyuPei22/NeuronSeek。
原文摘要 · Abstract (English)
Drawing inspiration from our human brain that designs different neurons for different tasks, recent advances in deep learning have explored modifying a network's neurons to develop so-called task-driven neurons. Prototyping task-driven neurons (referred to as NeuronSeek) employs symbolic regression (SR) to discover the optimal neuron formulation and construct a network from these optimized neurons. Along this direction, this work replaces symbolic regression with tensor decomposition (TD) to discover optimal neuronal formulations, offering enhanced stability and faster convergence. Furthermore, we establish theoretical guarantees that modifying the aggregation functions with common activation functions can empower a network with a fixed number of parameters to approximate any continuous function with an arbitrarily small error, providing a rigorous mathematical foundation for the NeuronSeek framework. Extensive empirical evaluations demonstrate that our NeuronSeek-TD framework not only achieves superior stability, but also is competitive relative to the state-of-the-art models across diverse benchmarks. The code is available at https://github.com/HanyuPei22/NeuronSeek.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。