提出按活动度动态剪枝的新方法，提升模型效率与精度
Budgeted Broadcast: An Activity-Dependent Pruning Rule for Neural Network Efficiency
- 基于单元活跃度和连接数分配本地流量预算，实现自适应剪枝
- 在多种模型上实现相同稀疏度下更高准确率，部分超越完整模型
- 适合追求高效推理的视觉、语音与医学图像任务研究者
现有剪枝方法多依据参数对损失的影响（如幅度或梯度）进行移除。本文提出预算广播（Budgeted Broadcast, BB），为每个单元分配本地流量预算（长期活跃率 $a_i$ 与扇出数 $k_i$ 的乘积）。通过约束熵分析发现，在全局流量预算下最大化编码熵可实现选择性与受众的平衡，即 $\log\frac{1-a_i}{a_i}=βk_i$。BB通过简单的局部机制实现该平衡：通过剪枝输入（降低活跃度）或输出（减少广播）。实验表明，BB在语音识别（Transformer）、人脸识别（ResNet）和突触预测（3D U-Net）中均提升了编码熵与去相关性，在匹配稀疏度下改善准确率，部分情况超过稠密基线。在电子显微图像上，其在我们评估协议下达到最先进的F1与PR-AUC。BB易于集成，为学习更丰富高效的表示提供了新路径。
原文摘要 · Abstract (English)
Most pruning methods remove parameters ranked by impact on loss (e.g., magnitude or gradient). We propose Budgeted Broadcast (BB), which gives each unit a local traffic budget (the product of its long-term on-rate $a_i$ and fan-out $k_i$). A constrained-entropy analysis shows that maximizing coding entropy under a global traffic budget yields a selectivity-audience balance, $\log\frac{1-a_i}{a_i}=βk_i$. BB enforces this balance with simple local actuators that prune either fan-in (to lower activity) or fan-out (to reduce broadcast). In practice, BB increases coding entropy and decorrelation and improves accuracy at matched sparsity across Transformers for ASR, ResNets for face identification, and 3D U-Nets for synapse prediction, sometimes exceeding dense baselines. On electron microscopy images, it attains state-of-the-art F1 and PR-AUC under our evaluation protocol. BB is easy to integrate and suggests a path toward learning more diverse and efficient representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。