通过输入扩展加速函数逼近,不增参数量却提升收敛与精度。
Hardware-Friendly Input Expansion for Accelerating Function Approximation
- 用常数扩展输入维度,打破参数对称性
- 平均减少12%的优化迭代次数,最优时误差降66.3%
- 适合需要高效推理的嵌入式或硬件部署场景
一维函数逼近是科学计算与工程应用中的基础问题。尽管神经网络具备强大通用逼近能力,但其优化过程常受参数空间对称性导致的平坦损失曲面阻碍,造成收敛慢、泛化差,尤其在高频成分上表现不佳。受物理中‘对称性破缺’启发,本文提出一种硬件友好的输入空间扩展方法:将原始一维输入(如 $x$)与常数(如 $π$)组合成高维向量(如 $[π, π, x, π, π]$),在不增加模型参数量的前提下有效打破参数对称性。我们在十类代表性一维函数(包括光滑、不连续、高频、非可导函数)上进行了评估。实验表明,输入空间扩展显著加快训练收敛(平均减少 LBFGS 迭代次数 12%),并提升逼近精度(最优 5D 扩展下最终均方误差降低 66.3%)。消融实验揭示不同扩展维度与常数选择的影响,$π$ 始终优于其他常数。本工作提出一种低成本、高效且硬件友好的算法设计技术。
原文摘要 · Abstract (English)
One-dimensional function approximation is a fundamental problem in scientific computing and engineering applications. While neural networks possess powerful universal approximation capabilities, their optimization process is often hindered by flat loss landscapes induced by parameter-space symmetries, leading to slow convergence and poor generalization, particularly for high-frequency components. Inspired by the principle of \emph{symmetry breaking} in physics, this paper proposes a hardware-friendly approach for function approximation through \emph{input-space expansion}. The core idea involves augmenting the original one-dimensional input (e.g., $x$) with constant values (e.g., $π$) to form a higher-dimensional vector (e.g., $[π, π, x, π, π]$), effectively breaking parameter symmetries without increasing the network's parameter count. We evaluate the method on ten representative one-dimensional functions, including smooth, discontinuous, high-frequency, and non-differentiable functions. Experimental results demonstrate that input-space expansion significantly accelerates training convergence (reducing LBFGS iterations by 12\% on average) and enhances approximation accuracy (reducing final MSE by 66.3\% for the optimal 5D expansion). Ablation studies further reveal the effects of different expansion dimensions and constant selections, with $π$ consistently outperforming other constants. Our work proposes a low-cost, efficient, and hardware-friendly technique for algorithm design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。