通过抑制频谱偏差，让神经网络权重的隐式表示更高效精准。
SBS: Enhancing Parameter-Efficiency of Neural Representations for Neural Networks via Spectral Bias Suppression
- 用双向排序平滑提升输出空间核的平滑性。
- 按层参数量自适应调节输入编码频率带宽，提升重建精度。
- 适合需要压缩模型参数又想保持高精度的场景。
隐式神经表示近年被用于以神经网络表示神经网络权重，带来显著参数压缩潜力。然而，标准多层感知机在神经网络表示中存在明显频谱偏差，难以有效重建高频细节。本文提出SBS，一种参数高效的改进方法，通过两种技术抑制频谱偏差：(1) 基于单向排序的平滑，提升输出空间中的核平滑性；(2) 基于单向排序的平滑感知随机傅里叶特征，根据层间参数量自适应调节输入编码的频率带宽。在CIFAR-10、CIFAR-100和ImageNet上的多种ResNet模型上进行广泛评估，结果表明SBS在参数更少的情况下，相比现有最先进方法实现了显著更高的重建精度。
原文摘要 · Abstract (English)
Implicit neural representations have recently been extended to represent convolutional neural network weights via neural representation for neural networks, offering promising parameter compression benefits. However, standard multi-layer perceptrons used in neural representation for neural networks exhibit a pronounced spectral bias, hampering their ability to reconstruct high-frequency details effectively. In this paper, we propose SBS, a parameter-efficient enhancement to neural representation for neural networks that suppresses spectral bias using two techniques: (1) a unidirectional ordering-based smoothing that improves kernel smoothness in the output space, and (2) unidirectional ordering-based smoothing aware random fourier features that adaptively modulate the frequency bandwidth of input encodings based on layer-wise parameter count. Extensive evaluations on various ResNet models with datasets CIFAR-10, CIFAR-100, and ImageNet, demonstrate that SBS achieves significantly better reconstruction accuracy with less parameters compared to SOTA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。