arXiv:2608.05464q-bio.NCcs.LG2026-08

用噪声波动指导神经网络剪枝,保留任务性能。

Effective pruning of task-trained recurrent neural networks using noisy fluctuations and connection rescaling

  • 基于连接噪声波动评估重要性,动态采样保留关键连接。
  • 剪枝后任务准确率几乎不变,优于仅看权重大小的方法。
  • 适合研究生物可解释的神经网络优化,尤其对递归网络。

网络连接剪枝对大脑功能至关重要,但目前缺乏兼具生物学合理性与良好性能的剪枝规则。本文评估了一种新提出的无监督局部剪枝方法——noise-prune,该方法利用噪声波动判断连接重要性。此前该方法仅在无特定计算任务的随机网络上测试过。我们证明,在任务训练过的循环神经网络中,noise-prune 能有效保持任务性能,显著优于仅依赖连接权重大小的策略,并达到或超过依赖二阶信息的非局部策略。noise-prune 不是直接剔除低于阈值的连接,而是根据重要性概率采样保留连接,并通过重标度强化保留连接以维持平均突触强度。我们发现这种采样与重标度机制对性能至关重要,但实际最优重标度程度低于原始理论预测值。本工作验证了 noise-prune 作为功能性循环网络架构的生物可解释剪枝规则的有效性,并明确了其最优参数设置。

原文摘要 · Abstract (English)

The pruning of network connections is key to brain function but, despite its importance, there exist few biologically-plausible pruning rules with demonstrated good performance. In this work we evaluate noise-prune, a recently introduced unsupervised local pruning rule for recurrent networks that uses noisy fluctuations to determine the importance of connections. Noise-prune has previously only been empirically tested on random networks without a specific computational function. We show that noise-prune preserves task-performance in task-trained recurrent neural networks, greatly outperforming a strategy that only uses the magnitude of connections and performing on par with or exceeding a non-local strategy that uses second-order information. Rather than deterministically removing connections that fall below a certain threshold importance, noise-prune samples connections to preserve based on their importance and strengthens retained connections to preserve average synaptic strength. We show that this sampling and rescaling is essential to good performance, but that the optimal empirical degree of rescaling is lower than that predicted by the original theoretical argument. Our work thus validates noise-prune as a biologically-plausible pruning rule for functional recurrent network architectures and characterizes its optimal parameter settings.

神经网络剪枝循环网络生物启发

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。