提出可显式建模空间变化频谱的神经表示方法，提升图像音频3D重建精度
NSTR: Neural Spectral Transport Representation for Space-Varying Frequency Fields
- 通过可学习的频谱传输方程，显式建模局部频谱随空间变化
- 在图像、音频和3D几何重建中实现更优的精度-参数权衡，减少全局频率数量
- 能可视化频谱流动，解释信号结构，适合需要高保真与可解释性的场景
隐式神经表示（INRs）已成为图像、音频和3D场景等信号表示的强大范式。然而，现有INR框架（如带傅里叶特征的MLP、SIREN、多分辨率哈希网格）隐含假设频谱基是全局且静态的，这与真实信号的空间频谱变化不一致——局部存在高频纹理、平滑区域及频率漂移现象。本文提出首个显式建模空间变化局部频谱场的INR框架：神经频谱传输表示（NSTR）。NSTR引入可学习的频谱传输方程（偏微分方程），描述局部频谱组成如何随空间演化。给定可学习的局部频谱场 $S(x)$ 与频谱传输网络 $F_θ$ 满足 $ abla S(x) ightarrow F_θ(x, S(x))$，NSTR通过空间调制一组紧凑的全局正弦基来重建信号。该框架实现强局部自适应性，并可通过可视化频谱流提供新层级的可解释性。在2D图像回归、音频重建和隐式3D几何任务上的实验表明，NSTR在精度-参数权衡上显著优于SIREN、傅里叶特征MLP和Instant-NGP，所需全局频率更少、收敛更快，并能自然解释信号结构。我们认为NSTR为INR研究开辟了新方向。
原文摘要 · Abstract (English)
Implicit Neural Representations (INRs) have emerged as a powerful paradigm for representing signals such as images, audio, and 3D scenes. However, existing INR frameworks -- including MLPs with Fourier features, SIREN, and multiresolution hash grids -- implicitly assume a \textit{global and stationary} spectral basis. This assumption is fundamentally misaligned with real-world signals whose frequency characteristics vary significantly across space, exhibiting local high-frequency textures, smooth regions, and frequency drift phenomena. We propose \textbf{Neural Spectral Transport Representation (NSTR)}, the first INR framework that \textbf{explicitly models a spatially varying local frequency field}. NSTR introduces a learnable \emph{frequency transport equation}, a PDE that governs how local spectral compositions evolve across space. Given a learnable local spectrum field $S(x)$ and a frequency transport network $F_θ$ enforcing $\nabla S(x) \approx F_θ(x, S(x))$, NSTR reconstructs signals by spatially modulating a compact set of global sinusoidal bases. This formulation enables strong local adaptivity and offers a new level of interpretability via visualizing frequency flows. Experiments on 2D image regression, audio reconstruction, and implicit 3D geometry show that NSTR achieves significantly better accuracy-parameter trade-offs than SIREN, Fourier-feature MLPs, and Instant-NGP. NSTR requires fewer global frequencies, converges faster, and naturally explains signal structure through spectral transport fields. We believe NSTR opens a new direction in INR research by introducing explicit modeling of space-varying spectrum.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。