提出一种无需搜索的层间自适应剪枝框架，自动分配每层剪枝比例。
FAIR-Pruner: A Flexible Framework for Automatic Layer-Wise Pruning via Tolerance of Difference
- 基于移除与保护双重信号，用差异容忍度动态决定各层剪枝深度。
- 在多个数据集和模型上实现高精度与高压缩比的平衡，最高压缩率超60%。
- 适用于视觉模型及专家路由架构，适合需要高效部署的工程师使用。
结构化剪枝是压缩深度神经网络的标准方法，但其实际性能取决于层间稀疏度的分配。本文提出 FAIR-Pruner，一种无需搜索的自适应层间结构化剪枝框架。该框架采用两种层内排序：面向移除的信号用于提出候选单元，面向保护的信号用于识别任务敏感单元。核心组件 Tolerance of Difference (ToD) 衡量移除前缀与保护尾部的重叠，并通过共享容差水平实现层间非均匀剪枝深度。默认视觉实例结合基于 Wasserstein 的 U-Score（用于类别条件单元可分性）与基于 Taylor 的 R-Score（用于任务级敏感性）；相同 ToD 分配规则也可搭配其他移除信号。理论上，我们通过群体 R-Score 分析 ToD，推导出高 R-Score 质量进入剪枝集的秩控制机制，并识别出与均匀剪枝同预算比较的加法交换条件。在 CIFAR-10、CIFAR-100、SVHN 以及 ImageNet 上，对 VGG、ResNet、DenseNet、ConvNeXt、DeiT 等模型的实验表明其具有出色的精度-压缩权衡能力。对路由专家模型 Qwen1.5-MoE-A2.7B-Chat 的纯剪枝实验进一步验证了其在相同专家预算下的架构可扩展性。FAIR-Pruner 已作为 pip 可安装的开源包发布。
原文摘要 · Abstract (English)
Structured pruning is a standard tool for compressing deep neural networks, but its practical performance depends on how sparsity is allocated across layers. We propose FAIR-Pruner, a search-free framework for adaptive layer-wise structured pruning. FAIR-Pruner uses two within-layer rankings: a removal-oriented signal that proposes candidate units and a protection-oriented signal that identifies task-sensitive units. Its core component, Tolerance of Difference (ToD), measures the overlap between the removal prefix and the protected tail, and uses a shared tolerance level to induce non-uniform pruning depths across layers. As a default vision instantiation, FAIR-Pruner combines a Wasserstein-based U-Score for class-conditional unit separability with a Taylor-based R-Score for task-level sensitivity; the same ToD allocation rule can also be paired with alternative removal signals. Theoretically, we analyze ToD through the population R-Score, derive rank-based control of the high-R-Score mass entering the pruning set, and identify an additive exchange condition for same-budget comparison with uniform pruning. Experiments on CIFAR-10, CIFAR-100, SVHN, and ImageNet across VGG, ResNet, DenseNet, ConvNeXt, and DeiT show strong accuracy--compression trade-offs. Prune-only experiments on routed-expert Qwen1.5-MoE-A2.7B-Chat further examine architectural extensibility under matched expert budgets. FAIR-Pruner is released as a pip-installable open-source package.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。