用特征值构造可解释的非线性模型,兼顾表达力与系数透明性。
The Spectral Neuron

- 输入通过矩阵仿射变换,输出取特征值实现非线性。
- 矩阵维数增大时表达力提升,特征值决定函数凸凹性。
- 适合需要可解释性的高阶建模任务,如金融或医疗预测。
随着机器学习模型复杂度增加,简单模型的系数透明性和函数形状可控性逐渐丧失。一端是线性模型,具有系数透明性但表达力有限;另一端是神经网络,表达力随规模增强但高度不透明。本文提出谱神经元(spectral neuron):形式为 $f(x)=λ_k (A_0 + A_1 x + ... + A_n x^n)$,其中 $A_0,...,A_n$ 为可学习的实对称矩阵。输入通过仿射矩阵函数进入,输出取其一个特征值。该模型非线性但非线性来源数学明确。当矩阵维度增大时,模型表达力增强,同时通过学习矩阵保持系数透明性。例如,极值特征值可生成凸或凹函数,半定约束保证单调性,对应特征空间刻画局部特征影响。研究了该模型族的系数透明性、特征影响边界与形状控制能力,并验证其可学习与可扩展性。结合多个数学领域的谱理论,系统分析其表达力、透明性、特征影响与形状控制特性。代码已公开于 https://github.com/alexshtf/spectral_neuron_paper。
原文摘要 · Abstract (English)
As machine learned models increase in complexity and expressive power, features of simpler models, such as intrinsic coefficient transparency and control over the shape of the modeled function are lost. On the one edge of the spectrum we have simple linear models that possess coefficient transparency, but have a limited expressive power. On the other edge we have neural networks, that have expressive power that improves with scaling, but are mostly opaque. In this work we develop the \emph{spectral neuron} concept: a scalar model given by $f(x)=λ_k (A_0 + A_1 x + ... + A_n x_n)$, with learned real symmetric matrices $A_0, ..., A_n$. The input enters the model through an affine matrix function, but the prediction is obtained by reading one of its eigenvalues. Thus, the model is nonlinear, but the source of nonlinearity is still mathematically explicit. This gives us a useful middle ground: the model can become more expressive as the matrix dimension grows, while retaining coefficient transparency through the learned matrices. For example, extremal eigenvalues yield convex or concave functions, semidefinite constraints on the coefficient matrices impose monotonicity, and the associated eigenspaces characterize local feature influence. We study coefficient transparency, feature-influence bounds, and shape-control properties of this model family, and then test whether it can be learned and scaled in practice. We develop a systematic study of this model family, bringing together spectral results from several mathematical literatures to characterize its expressivity, coefficient transparency, feature influence, and shape-control properties. Code available at https://github.com/alexshtf/spectral_neuron_paper.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。