提出双向剪枝再生策略，突破高稀疏下精度骤降瓶颈
Beyond One-Way Pruning: Bidirectional Pruning-Regrowth for Extreme Accuracy-Sparsity Tradeoff
- 从极稀疏网络出发，选择性恢复关键连接
- 在高稀疏度下保持模型精度，避免性能陡降
- 适合资源受限设备部署的极致压缩场景
模型剪枝作为广泛采用的模型压缩技术，在多种架构中表现出色。然而我们发现，当稀疏度超过某一阈值时，迭代和一次性剪枝方法均导致模型性能急剧下降。这种快速退化限制了可实现的压缩比，使模型无法满足某些硬件平台的严苛尺寸约束，进而无法运行。为克服此限制，我们提出一种双向剪枝-再生策略：从满足硬件约束的极稀疏网络出发，选择性地恢复关键连接以恢复损失的性能，有效缓解高稀疏条件下常见的精度急剧下降问题。
原文摘要 · Abstract (English)
As a widely adopted model compression technique, model pruning has demonstrated strong effectiveness across various architectures. However, we observe that when sparsity exceeds a certain threshold, both iterative and one-shot pruning methods lead to a steep decline in model performance. This rapid degradation limits the achievable compression ratio and prevents models from meeting the stringent size constraints required by certain hardware platforms, rendering them inoperable. To overcome this limitation, we propose a bidirectional pruning-regrowth strategy. Starting from an extremely compressed network that satisfies hardware constraints, the method selectively regenerates critical connections to recover lost performance, effectively mitigating the sharp accuracy drop commonly observed under high sparsity conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。