调控训练中的网络秩，让普通MLP也能高保真表示高频信号。
Optimizing Rank for High-Fidelity Implicit Neural Representations
- 用优化器控制训练时的网络秩，避免低频偏差
- 在图像和新视角合成任务中最高提升9 dB PSNR
- 适合想提升隐式神经表示质量的研究者
基于原始多层感知机（MLP）的隐式神经表示（INRs）普遍被认为无法表达高频内容，这促使研究转向坐标嵌入或特殊激活函数等架构改进。本文挑战这一观点，指出低频偏好实为训练中稳定秩退化所致，而非架构固有缺陷。实验表明，通过调控训练过程中的网络秩，显著提升信号重建保真度，使简单MLP也具备强表达能力。使用如Muon等具有高秩、近正交更新的优化器，在自然图像、医学影像及新视角合成等多个领域均实现显著性能提升，相较相同架构最高达+9 dB PSNR。代码已公开于https://rank-inrs.github.io。
原文摘要 · Abstract (English)
Implicit Neural Representations (INRs) based on vanilla Multi-Layer Perceptrons (MLPs) are widely believed to be incapable of representing high-frequency content. This has directed research efforts towards architectural interventions, such as coordinate embeddings or specialized activation functions, to represent high-frequency signals. In this paper, we challenge the notion that the low-frequency bias of vanilla MLPs is an intrinsic, architectural limitation to learn high-frequency content, but instead a symptom of stable rank degradation during training. We empirically demonstrate that regulating the network's rank during training substantially improves the fidelity of the learned signal, rendering even simple MLP architectures expressive. Extensive experiments show that using optimizers like Muon, with high-rank, near-orthogonal updates, consistently enhances INR architectures even beyond simple ReLU MLPs. These substantial improvements hold across a diverse range of domains, including natural and medical images and novel view synthesis, with up to +9 dB PSNR over the same architecture. Code is available at (https://rank-inrs.github.io).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。