BINGO一次性剪枝，高效缩小模型且不降精度。
BINGO: A Novel Pruning Mechanism to Reduce the Size of Neural Networks
- 训练中分批评估权重重要性，生成全局显著性评分
- 单次剪枝即可移除冗余参数，保持模型准确率
- 适合资源有限但需高效压缩模型的研究与应用
过去十年，机器学习的使用呈指数级增长。模型变得空前复杂，规模庞大，包含数百万个参数。然而，大型模型成为主流，导致训练和运行成本高达数百万美元，不仅加重企业负担，也阻碍非富裕群体参与创新，并推高消费者价格。现有剪枝方法如迭代幅度剪枝虽能保持较高精度，但需反复训练，计算和环境成本极高。为此，本文提出BINGO：在训练过程中，分批次分析神经网络的特定子集，评估每个权重对模型准确率的贡献程度。训练结束后，为每个权重生成显著性评分，实现一次性的、精度保持的剪枝。该方法显著降低计算开销，使人工智能发展不必依赖模型膨胀。
原文摘要 · Abstract (English)
Over the past decade, the use of machine learning has increased exponentially. Models are far more complex than ever before, growing to gargantuan sizes and housing millions of weights. Unfortunately, the fact that large models have become the state of the art means that it often costs millions of dollars to train and operate them. These expenses not only hurt companies but also bar non-wealthy individuals from contributing to new developments and force consumers to pay greater prices for AI. Current methods used to prune models, such as iterative magnitude pruning, have shown great accuracy but require an iterative training sequence that is incredibly computationally and environmentally taxing. To solve this problem, BINGO is introduced. BINGO, during the training pass, studies specific subsets of a neural network one at a time to gauge how significant of a role each weight plays in contributing to a network's accuracy. By the time training is done, BINGO generates a significance score for each weight, allowing for insignificant weights to be pruned in one shot. BINGO provides an accuracy-preserving pruning technique that is less computationally intensive than current methods, allowing for a world where AI growth does not have to mean model growth, as well.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。