用有理函数建模神经网络非线性，提升可解释性和外推能力
Rational ANOVA Networks
- 基于ANOVA分解与有理逼近，用可学习有理单元建模主效应和交互项
- 在相同参数量下，对CIFAR-10等任务性能优于或匹配MLP和可学习激活基线
- 正分母设计避免奇点，提升数值稳定性和模型吞吐，适合追求可解释性的场景
深度神经网络通常将非线性视为固定原语（如ReLU），限制了可解释性与函数类的控制粒度。尽管近期的加法模型（如KAN）采用样条进行改进，但常面临计算效率低和边界不稳定的缺陷。我们提出有理-ANOVA网络（RAN），其基础为函数ANOVA分解与帕德式有理逼近。RAN将函数f(x)建模为主效应与稀疏成对交互的组合，每个组件由稳定、可学习的有理单元参数化。关键在于强制分母严格为正，避免极点与数值不稳定，更高效地捕捉陡变与近奇异行为。该ANOVA结构为数据效率与可解释性提供显式低阶交互偏置，而有理参数化显著提升外推性能。在受控函数基准与视觉分类任务（如CIFAR-10）中，当参数与计算预算相当时，RAN表现匹配或超越参数匹配的MLP与可学习激活基线，且具备更好稳定性与吞吐。代码已开源。
原文摘要 · Abstract (English)
Deep neural networks typically treat nonlinearities as fixed primitives (e.g., ReLU), limiting both interpretability and the granularity of control over the induced function class. While recent additive models (like KANs) attempt to address this using splines, they often suffer from computational inefficiency and boundary instability. We propose the Rational-ANOVA Network (RAN), a foundational architecture grounded in functional ANOVA decomposition and Padé-style rational approximation. RAN models f(x) as a composition of main effects and sparse pairwise interactions, where each component is parameterized by a stable, learnable rational unit. Crucially, we enforce a strictly positive denominator, which avoids poles and numerical instability while capturing sharp transitions and near-singular behaviors more efficiently than polynomial bases. This ANOVA structure provides an explicit low-order interaction bias for data efficiency and interpretability, while the rational parameterization significantly improves extrapolation. Across controlled function benchmarks and vision classification tasks (e.g., CIFAR-10) under matched parameter and compute budgets, RAN matches or surpasses parameter-matched MLPs and learnable-activation baselines, with better stability and throughput. Code is available at https://github.com/jushengzhang/Rational-ANOVA-Networks.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。