揭示神经网络隐式表示中高频细节恢复难的根源，提出统一解释框架。
Understanding NTK Variance in Implicit Neural Representations
- 通过分析输入相似性与缩放项，揭示影响NTK特征值方差的关键机制。
- 证明位置编码、球归一化等方法可降低方差，提升收敛速度与重建质量。
- 适用于研究隐式表示、谱偏差优化或设计高效神经网络的读者。
隐式神经表示（INRs）常因谱偏差导致收敛慢且难以恢复高频细节。尽管已有研究将此现象与神经正切核（NTK）相关联，但具体架构选择如何影响NTK条件数仍不清晰。本文表明，多种INR机制可通过其对一组有限的成对相似性因子和缩放项的影响来理解，这些因子共同决定NTK特征值方差。对于标准坐标MLP，输入特征间交互受限会导致特征值离散度大、条件数差。我们推导了常见INR组件的闭式方差分解：位置编码重塑输入相似性，球归一化通过逐层缩放降低方差，哈达玛调制引入严格小于1的额外相似性因子，实现乘法式方差缩减。该统一视角解释了不同INR架构如何通过改善NTK条件数缓解谱偏差。多任务实验验证了预测的方差降低，并展示更快更稳定的收敛及更优重建效果。
原文摘要 · Abstract (English)
Implicit Neural Representations (INRs) often converge slowly and struggle to recover high-frequency details due to spectral bias. While prior work links this behavior to the Neural Tangent Kernel (NTK), how specific architectural choices affect NTK conditioning remains unclear. We show that many INR mechanisms can be understood through their impact on a small set of pairwise similarity factors and scaling terms that jointly determine NTK eigenvalue variance. For standard coordinate MLPs, limited input-feature interactions induce large eigenvalue dispersion and poor conditioning. We derive closed-form variance decompositions for common INR components and show that positional encoding reshapes input similarity, spherical normalization reduces variance via layerwise scaling, and Hadamard modulation introduces additional similarity factors strictly below one, yielding multiplicative variance reduction. This unified view explains how diverse INR architectures mitigate spectral bias by improving NTK conditioning. Experiments across multiple tasks confirm the predicted variance reductions and demonstrate faster, more stable convergence with improved reconstruction quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。