把剪枝看作模型组件的博弈均衡,自然产生稀疏性。
Pruning as a Game: Equilibrium-Driven Sparsification of Neural Networks
- 将网络参数视为博弈玩家,通过参与度平衡贡献与冗余。
- 在均衡下,表现差的组件自动归零,实现自然剪枝。
- 无需重要性评分,可解释性强,适合理论研究者。
神经网络剪枝广泛用于降低模型规模和计算开销。然而,现有方法大多将稀疏性作为外部强加的约束,依赖启发式重要性评分或训练时正则化。本文提出全新视角:剪枝是模型组件间策略互动的均衡结果。我们将权重、神经元或卷积核等参数组建模为连续非合作博弈中的玩家,每个玩家选择自身参与度以权衡贡献、冗余与竞争。在此框架下,当持续参与成为占优策略时,稀疏性自然出现。我们分析该博弈并证明,在温和条件下,被占优的玩家会趋于零参与,为剪枝行为提供理论解释。基于此,我们设计了一种简单且基于均衡的剪枝算法,联合更新网络参数与参与变量,无需显式重要性评分。本工作聚焦于建立剪枝作为均衡现象的原理性框架及实证验证,而非全面架构或大规模基准测试。标准基准实验表明,所提方法在稀疏性-准确率权衡上表现优异,且提供了可解释、理论基础扎实的剪枝替代方案。
原文摘要 · Abstract (English)
Neural network pruning is widely used to reduce model size and computational cost. Yet, most existing methods treat sparsity as an externally imposed constraint, enforced through heuristic importance scores or training-time regularization. In this work, we propose a fundamentally different perspective: pruning as an equilibrium outcome of strategic interaction among model components. We model parameter groups such as weights, neurons, or filters as players in a continuous non-cooperative game, where each player selects its level of participation in the network to balance contribution against redundancy and competition. Within this formulation, sparsity emerges naturally when continued participation becomes a dominated strategy at equilibrium. We analyze the resulting game and show that dominated players collapse to zero participation under mild conditions, providing a principled explanation for pruning behavior. Building on this insight, we derive a simple equilibrium-driven pruning algorithm that jointly updates network parameters and participation variables without relying on explicit importance scores. This work focuses on establishing a principled formulation and empirical validation of pruning as an equilibrium phenomenon, rather than exhaustive architectural or large-scale benchmarking. Experiments on standard benchmarks demonstrate that the proposed approach achieves competitive sparsity-accuracy trade-offs while offering an interpretable, theory-grounded alternative to existing pruning methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。