提出一种适用于各类激活函数的神经网络初始化方法，提升隐式表示模型训练稳定性与性能。
VI3NR: Variance Informed Initialization for Implicit Neural Representations
- 基于方差稳定原则，推导出适配任意激活函数的初始化策略。
- 在图像、音频、3D表面重建任务中均实现性能提升，尤其对高斯隐式表示效果显著。
- 理论与实验一致，适合研究隐式神经表征的开发者使用。
隐式神经表示（INRs）是编码图像、视频、声音和3D形状等多种数据的强大工具。网络初始化对INRs的成功至关重要，但现有初始化方法不适用于多数激活函数，尤其是INRs常用的类型。本文提出一种方差稳定的初始化方法，适用于任意激活函数，可统一多种已有初始化方案，并在经典激活函数上表现更优。实验表明，该方法在多模态信号重建中显著提升性能，尤其在高斯隐式表示中，理论预测与实际表现高度一致，实现了图像、音频和3D表面重建的改进。
原文摘要 · Abstract (English)
Implicit Neural Representations (INRs) are a versatile and powerful tool for encoding various forms of data, including images, videos, sound, and 3D shapes. A critical factor in the success of INRs is the initialization of the network, which can significantly impact the convergence and accuracy of the learned model. Unfortunately, commonly used neural network initializations are not widely applicable for many activation functions, especially those used by INRs. In this paper, we improve upon previous initialization methods by deriving an initialization that has stable variance across layers, and applies to any activation function. We show that this generalizes many previous initialization methods, and has even better stability for well studied activations. We also show that our initialization leads to improved results with INR activation functions in multiple signal modalities. Our approach is particularly effective for Gaussian INRs, where we demonstrate that the theory of our initialization matches with task performance in multiple experiments, allowing us to achieve improvements in image, audio, and 3D surface reconstruction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。