动态调整神经网络结构,让模型更小且更准。
Adaptive Training of INRs via Pruning and Densification
- 训练中自动剪枝冗余神经元,提升效率。
- 在信号欠拟合区域添加新频率,增强表达能力。
- 适合追求轻量化与高精度的图像与形状建模任务。
将正弦函数编码输入坐标嵌入多层感知机(MLPs)已被证明对低维信号的隐式神经表示(INRs)有效,能够建模高频细节。然而,如何选择合适的输入频率和网络结构以管理参数冗余仍是开放挑战,通常依赖启发式方法和复杂的超参数优化。本文提出AIRe(自适应隐式神经表示),一种在优化过程中动态调整INR架构的训练方案。该方法通过神经元剪枝避免冗余,并通过输入频率密集化提升表示能力,从而在模型规模与重建质量之间取得更好平衡。剪枝阶段首先识别贡献度较低的神经元,施加定向权重衰减以将其信息转移至保留神经元,随后进行结构化剪枝;密度扩展阶段则向信号欠拟合的频谱区域添加输入频率,扩大表示基。在图像与符号距离场(SDFs)上的实验表明,AIRe在减少模型大小的同时,保持甚至提升了重建质量。代码与预训练模型将公开发布。
原文摘要 · Abstract (English)
Encoding input coordinates with sinusoidal functions into multilayer perceptrons (MLPs) has proven effective for implicit neural representations (INRs) of low-dimensional signals, enabling the modeling of high-frequency details. However, selecting appropriate input frequencies and architectures while managing parameter redundancy remains an open challenge, often addressed through heuristics and heavy hyperparameter optimization schemes. In this paper, we introduce AIRe ($\textbf{A}$daptive $\textbf{I}$mplicit neural $\textbf{Re}$presentation), an adaptive training scheme that refines the INR architecture over the course of optimization. Our method uses a neuron pruning mechanism to avoid redundancy and input frequency densification to improve representation capacity, leading to an improved trade-off between network size and reconstruction quality. For pruning, we first identify less-contributory neurons and apply a targeted weight decay to transfer their information to the remaining neurons, followed by structured pruning. Next, the densification stage adds input frequencies to spectrum regions where the signal underfits, expanding the representational basis. Through experiments on images and SDFs, we show that AIRe reduces model size while preserving, or even improving, reconstruction quality. Code and pretrained models will be released for public use.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。