用Transformer直接求解病态线性方程组，省去传统方法的调参和预处理。
Algebraformer: A Neural Approach to Linear Systems
- 基于Transformer设计新编码方式，实现矩阵向量高效表示。
- 在边界值问题插值与牛顿法加速任务中精度媲美传统方法。
- 测试时计算开销显著降低，适合科学计算场景部署。
深度学习为解决经典算法任务提供了新可能。本文研究求解线性系统这一基础问题，特别是病态情形。现有数值方法对病态系统常需精细调参、预处理或领域专长以保证精度与稳定性。我们提出Algebraformer，一种基于Transformer的端到端架构，可直接求解严重病态的线性系统。模型采用新型编码方案，实现矩阵与向量输入的高效表示，内存复杂度为$O(n^2)$，支持可扩展推理。我们在应用驱动的线性问题上验证其有效性，包括谱方法求解边值问题的插值任务以及牛顿法加速。Algebraformer在测试时达到与传统方法相当的精度，但计算开销显著更低，表明通用神经架构能有效简化传统科学计算流程。
原文摘要 · Abstract (English)
Recent work in deep learning has opened new possibilities for solving classical algorithmic tasks using end-to-end learned models. In this work, we investigate the fundamental task of solving linear systems, particularly those that are ill-conditioned. Existing numerical methods for ill-conditioned systems often require careful parameter tuning, preconditioning, or domain-specific expertise to ensure accuracy and stability. In this work, we propose Algebraformer, a Transformer-based architecture that learns to solve linear systems end-to-end, even in the presence of severe ill-conditioning. Our model leverages a novel encoding scheme that enables efficient representation of matrix and vector inputs, with a memory complexity of $O(n^2)$, supporting scalable inference. We demonstrate its effectiveness on application-driven linear problems, including interpolation tasks from spectral methods for boundary value problems and acceleration of the Newton method. Algebraformer achieves competitive accuracy with significantly lower computational overhead at test time, demonstrating that general-purpose neural architectures can effectively reduce complexity in traditional scientific computing pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。