用神经网络和Transformer近似矩阵函数，提升科学计算效率。
Approximating Matrix Functions with Deep Neural Networks and Transformers
- 用ReLU网络逼近矩阵指数，给出深度与宽度的理论上限。
- Transformer在合适编码下，对部分矩阵函数相对误差低至5%。
- 编码方式显著影响效果，不同函数需适配不同编码策略。
Transformer在自然语言处理中已取得突破，但在数值计算中的应用仍较少。本文研究使用神经网络（包括Transformer）逼近矩阵函数的能力，这些函数将标量函数映射为同维方阵。此类函数广泛出现在科学计算中，如连续时间马尔可夫链中的矩阵指数、动力系统稳定性分析中的矩阵符号函数。本文做出两项贡献：首先，证明了任意精度逼近矩阵指数所需的ReLU网络的宽度与深度上界；其次，实验表明，采用适当数值编码的Transformer编码器-解码器架构，能在高概率下以5%的相对误差逼近特定矩阵函数。研究揭示，编码方案对性能有显著影响，不同函数需匹配不同的编码策略。
原文摘要 · Abstract (English)
Transformers have revolutionized natural language processing, but their use for numerical computation has received less attention. We study the approximation of matrix functions, which map scalar functions to matrices, using neural networks including transformers. We focus on functions mapping square matrices to square matrices of the same dimension. These types of matrix functions appear throughout scientific computing, e.g., the matrix exponential in continuous-time Markov chains and the matrix sign function in stability analysis of dynamical systems. In this paper, we make two contributions. First, we prove bounds on the width and depth of ReLU networks needed to approximate the matrix exponential to an arbitrary precision. Second, we show experimentally that a transformer encoder-decoder with suitable numerical encodings can approximate certain matrix functions at a relative error of 5% with high probability. Our study reveals that the encoding scheme strongly affects performance, with different schemes working better for different functions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。