用神经场建模声音,提出新框架提升音频表示稳定性。
Representing Sounds as Neural Amplitude Fields: A Benchmark of Coordinate-MLPs and A Fourier Kolmogorov-Arnold Framework
- 基于傅里叶与柯尔莫哥洛夫定理设计新网络,无需额外位置编码。
- 在语音和音乐数据集上实现无需调参的高质量音频重建。
- 适合音频生成、压缩等需要连续表示的任务开发者参考。
尽管基于坐标MLP的隐式神经表示在辐射场、3D形状和图像领域表现优异,但其在音频信号中的应用仍不充分。为填补这一空白,我们系统研究了现有隐式表示方法,提取出3种位置编码和16种常用激活函数,并通过组合设计建立了首个用于音频信号的Coordinate-MLP基准。实验表明,Coordinate-MLP需复杂超参数调优和频率相关初始化,限制了其鲁棒性。为此,我们提出Fourier-ASR框架,结合傅里叶级数定理与柯尔莫哥洛夫-阿诺德表示定理,引入傅里叶-柯尔莫哥洛夫网络(Fourier-KAN),利用周期性与强非线性特性表示音频信号,无需额外位置编码。同时提出频适应学习策略(FaLS),有效捕捉高频成分并防止低频过拟合。在自然语音与音乐数据集上的大量实验表明:(1) 合理设计的位置编码与激活函数可显著提升音频表示质量;(2) Fourier-ASR可在无需精细调参的情况下稳健表示复杂音频信号。未来,隐式音频表示的连续性与无限分辨率特性使其在音频压缩、合成与生成任务中极具潜力。代码已开源:https://github.com/lif314/Fourier-ASR。
原文摘要 · Abstract (English)
Although Coordinate-MLP-based implicit neural representations have excelled in representing radiance fields, 3D shapes, and images, their application to audio signals remains underexplored. To fill this gap, we investigate existing implicit neural representations, from which we extract 3 types of positional encoding and 16 commonly used activation functions. Through combinatorial design, we establish the first benchmark for Coordinate-MLPs in audio signal representations. Our benchmark reveals that Coordinate-MLPs require complex hyperparameter tuning and frequency-dependent initialization, limiting their robustness. To address these issues, we propose Fourier-ASR, a novel framework based on the Fourier series theorem and the Kolmogorov-Arnold representation theorem. Fourier-ASR introduces Fourier Kolmogorov-Arnold Networks (Fourier-KAN), which leverage periodicity and strong nonlinearity to represent audio signals, eliminating the need for additional positional encoding. Furthermore, a Frequency-adaptive Learning Strategy (FaLS) is proposed to enhance the convergence of Fourier-KAN by capturing high-frequency components and preventing overfitting of low-frequency signals. Extensive experiments conducted on natural speech and music datasets reveal that: (1) well-designed positional encoding and activation functions in Coordinate-MLPs can effectively improve audio representation quality; and (2) Fourier-ASR can robustly represent complex audio signals without extensive hyperparameter tuning. Looking ahead, the continuity and infinite resolution of implicit audio representations make our research highly promising for tasks such as audio compression, synthesis, and generation. The source code will be released publicly to ensure reproducibility. The code is available at https://github.com/lif314/Fourier-ASR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。