固定结构的神经网络可通过运行时间逼近任意连续函数。
Recurrent neural networks approximate continuous functions
- 用单个固定权重的RNN，靠时间推演逼近函数
- 收敛速度与多项式逼近速率一致，可证明最优
- 适合研究计算资源与模型精度关系的学者
经典逼近定理要求每次提升精度都要重新设计网络。本文探讨相反可能：能否仅用一个固定结构的网络，通过延长运行时间来提高精度？我们证明，对于区间[-1,1]上的任意连续函数，都存在一个固定权重、固定隐藏维度的ReLU RNN，其随时间演化能一致逼近该函数。核心机制是一种新型中间模型——带有神经单元的图灵机（TMNU），它既保持算法灵活性以实现多项式逼近，又足够紧凑，可被带显式约束的RNN模拟。收敛速率反映底层多项式逼近率。我们还给出极小极大下界，证明运行时间并非理论构造的产物，而是此固定网络范式中不可或缺的资源。
原文摘要 · Abstract (English)
Classical approximation theorems ask for a new neural network whenever the target accuracy is improved. This paper studies the opposite possibility: can the network be chosen once and for all, and can accuracy be bought only by letting it run longer? We prove that this is possible for every continuous function on [-1,1]. More precisely, each such function is uniformly approximated by the time evolution of a single ReLU recurrent neural network with fixed weights and fixed hidden dimension. The mechanism behind the construction is a new intermediate model, the Turing machine with neural units (TMNU). This model retains the algorithmic freedom needed to implement polynomial approximation schemes, while remaining rigid enough to be simulated by RNNs with explicit bounds on hidden dimension and weight magnitude. The resulting convergence rates reflect the underlying polynomial approximation rates. We complement the construction with minimax lower bounds showing that runtime is not merely a proof artifact, but an unavoidable resource in this fixed-network approximation paradigm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。