arXiv:2602.06374math.FAcs.LG2026-02

提出乘法神经网络架构,提升对不规则函数的逼近精度。

A Multiplicative Neural Network Architecture: Locality and Regularity of Approximation

  • 用乘法交互代替加法构建网络基础表示
  • 在奇异层区域误差更集中,正则性度量收敛更稳定
  • 适合处理带尖锐过渡或高阶光滑性丧失的问题

我们提出一种以乘法交互为核心表示的神经网络架构,而非作为加法模型中的辅助组件。建立了该架构的通用逼近定理,并在贝塞尔势空间中分析其逼近的局部性和正则性。为补充理论结果,我们在具有尖锐过渡层或点状高阶正则性损失的典型目标上进行数值实验。实验聚焦于逼近误差的空间结构及正则性敏感量,特别是Zygmund型半范数的收敛性。结果表明,所提乘法架构产生的残差结构更紧密对齐于正则性降低区域,且在正则性敏感指标上表现出更稳定的收敛性。这些结果证明,采用乘法表示形式对神经网络逼近的定位性与正则性行为有明确影响,直接建立了架构设计与逼近函数解析性质之间的联系。

原文摘要 · Abstract (English)

We introduce a multiplicative neural network architecture in which multiplicative interactions constitute the fundamental representation, rather than appearing as auxiliary components within an additive model. We establish a universal approximation theorem for this architecture and analyze its approximation properties in terms of locality and regularity in Bessel potential spaces. To complement the theoretical results, we conduct numerical experiments on representative targets exhibiting sharp transition layers or pointwise loss of higher-order regularity. The experiments focus on the spatial structure of approximation errors and on regularity-sensitive quantities, in particular, the convergence of Zygmund-type seminorms. The results show that the proposed multiplicative architecture yields residual error structures that are more tightly aligned with regions of reduced regularity and exhibit more stable convergence in regularity-sensitive metrics. These results demonstrate that adopting a multiplicative representation format has concrete implications for the localization and regularity behavior of neural network approximations, providing a direct connection between architectural design and analytical properties of the approximating functions.

神经网络架构逼近理论正则性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。