arXiv:2410.01990cs.LGcs.CE2024-10ICLR被引 18

提出ActNet模型,用新方法改进柯尔莫哥洛夫定理在神经网络中的应用

Deep Learning Alternatives of the Kolmogorov Superposition Theorem

  • 基于柯尔莫哥洛夫超位置定理设计可扩展的深层网络结构
  • 在物理信息神经网络中优于KAN,逼近偏微分方程表现更优
  • 适合科学计算与偏微分方程模拟任务,对低维函数逼近有优势

本文探索柯尔莫哥洛夫超位置定理(KST)的替代形式,作为神经网络设计的基础。原始KST虽数学优美,但因对内外函数结构缺乏洞察且引入大量未知变量,实用性受限。柯尔莫哥洛夫-阿诺德网络(KANs)虽利用KST进行函数逼近,但与传统多层感知机(MLPs)相比结果参差,且受原形式制约。为此,本文提出ActNet,一种基于KST的可扩展深度学习模型,克服了原始形式的诸多缺陷。在物理信息神经网络(PINNs)框架下评估,该框架擅长利用KST在低维函数逼近上的优势,尤其适用于求解偏微分方程(PDEs)。在此高挑战性场景中,模型需在无直接观测数据情况下学习隐含函数,ActNet在多个基准测试中持续优于KANs,且与当前最优的基于MLP的方法相当。这些结果表明,ActNet为基于KST的深度学习应用提供了有前景的新方向,尤其在科学计算与PDE模拟任务中具有潜力。

原文摘要 · Abstract (English)

This paper explores alternative formulations of the Kolmogorov Superposition Theorem (KST) as a foundation for neural network design. The original KST formulation, while mathematically elegant, presents practical challenges due to its limited insight into the structure of inner and outer functions and the large number of unknown variables it introduces. Kolmogorov-Arnold Networks (KANs) leverage KST for function approximation, but they have faced scrutiny due to mixed results compared to traditional multilayer perceptrons (MLPs) and practical limitations imposed by the original KST formulation. To address these issues, we introduce ActNet, a scalable deep learning model that builds on the KST and overcomes many of the drawbacks of Kolmogorov's original formulation. We evaluate ActNet in the context of Physics-Informed Neural Networks (PINNs), a framework well-suited for leveraging KST's strengths in low-dimensional function approximation, particularly for simulating partial differential equations (PDEs). In this challenging setting, where models must learn latent functions without direct measurements, ActNet consistently outperforms KANs across multiple benchmarks and is competitive against the current best MLP-based approaches. These results present ActNet as a promising new direction for KST-based deep learning applications, particularly in scientific computing and PDE simulation tasks.

深度学习柯尔莫哥洛夫定理科学计算神经网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。