通过反向再生突破模型压缩极限,实现稳定高效剪枝。
Breaking the Compression Barrier: Cross-Architecture Compression Boundary Learning via Reverse Regrowth

- 先极稀疏化再选择性恢复关键结构,重构压缩边界。
- 在非结构化剪枝中提升1.49%,结构化剪枝中提升4.77%。
- 适用于CNN与Transformer,适合边缘设备部署场景。
模型压缩对资源受限的边缘设备部署至关重要。尽管基于剪枝的方法能显著减小模型规模,但通常在稀疏度超过阈值后出现性能骤降,难以确定可行压缩极限。为此,我们提出边界学习反向再生框架BRIDGE,将压缩重构为构造性边界搜索问题。不同于正向剪枝,该方法先将模型驱动至极度稀疏状态以暴露崩溃区域,再选择性地恢复关键结构以恢复性能。框架采用分层再生策略,包括粗粒度层选择和细粒度参数选择,精准识别需恢复的参数。实验表明,该方法可在CNN与Transformer架构上从崩溃边缘恢复模型性能,具备架构无关性。在非结构化剪枝中性能提升达1.49%,结构化剪枝中最高提升4.77%。结果表明,反向再生可有效拓展压缩极限并保持性能稳定。代码已开源:https://github.com/EnumaCaliber/BRIDGE。
原文摘要 · Abstract (English)
Model compression is critical for deploying networks on resource-constrained edge devices. While pruning-based methods can significantly reduce model size, they often suffer from abrupt performance collapse beyond a sparsity thresh-old, making it difficult to identify the feasible compression limit of the model. To address this challenge, we propose a boundary-Learning reverse regrowth framework, BRIDGE, that reformulates compression as a constructive boundary-search problem. Unlike forward pruning, our method first drives the model to an extremely sparse state to expose the collapse region, and then selectively regenerates the critical structure to restore performance. The proposed framework employs a hierarchical regeneration strategy, including coarse-grained layer selection and fine-grained regeneration parameter selection, to accurately identify which parameters require recovery. Experiments show that our method can recover models from the brink of collapse on both CNNs and Transformer architectures, demonstrating its architecture in-dependence. BRIDGE achieves a performance improvement of up to 1.49% in unstructured pruning and up to 4.77% in structured pruning. These results demonstrate that reverse regeneration can effectively extend the compression limit while maintaining stable performance. The source code is available at https://github.com/EnumaCaliber/BRIDGE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。