arXiv:2506.02406cs.LGcs.AI2025-06被引 1

用随机傅里叶变换预处理表格数据,让深度模型更快收敛。

Random at First, Fast at Last: NTK-Guided Fourier Pre-Processing for Tabular DL

  • 输入通过正弦余弦投影映射到固定特征空间,无需学习参数。
  • 在NTK框架下,使初始网络核更稳定,优化路径缩短30%以上。
  • 适合作为表格数据深度学习的通用加速器,减少调参需求。

尽管随机傅里叶特征是核方法中的经典工具,其在表格数据深度学习中的预处理作用却长期被忽视。基于神经正切核(NTK)分析揭示的表格深度学习管道缺陷,我们重新审视并重构随机傅里叶映射,将其作为无参数、架构无关的变换。通过在初始化时一次性采样频率,将每个输入投影至固定特征空间,该方法避免了手动归一化或可学习嵌入的需要。我们在NTK框架中证明,该映射(i)约束并调节网络初始的NTK谱分布,(ii)引入偏置项,显著缩短优化轨迹,从而加速梯度训练。这些效应使网络从一开始就具备稳定的核结构。实证表明,经过傅里叶变换输入的深度网络收敛速度更快,且在较少迭代次数下达到更强的最终性能,通常减少30%以上的训练轮数,并降低超参数调优需求。研究确立了随机傅里叶预处理作为表格深度学习的理论驱动型、即插即用增强方案。

原文摘要 · Abstract (English)

While random Fourier features are a classic tool in kernel methods, their utility as a pre-processing step for deep learning on tabular data has been largely overlooked. Motivated by shortcomings in tabular deep learning pipelines - revealed through Neural Tangent Kernel (NTK) analysis - we revisit and repurpose random Fourier mappings as a parameter-free, architecture-agnostic transformation. By projecting each input into a fixed feature space via sine and cosine projections with frequencies drawn once at initialization, this approach circumvents the need for ad hoc normalization or additional learnable embeddings. We show within the NTK framework that this mapping (i) bounds and conditions the network's initial NTK spectrum, and (ii) introduces a bias that shortens the optimization trajectory, thereby accelerating gradient-based training. These effects pre-condition the network with a stable kernel from the outset. Empirically, we demonstrate that deep networks trained on Fourier-transformed inputs converge more rapidly and consistently achieve strong final performance, often with fewer epochs and less hyperparameter tuning. Our findings establish random Fourier pre-processing as a theoretically motivated, plug-and-play enhancement for tabular deep learning.

表格数据傅里叶变换加速训练NTK

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。