用混合QUBO优化实现更精准的神经网络剪枝,兼顾滤波器重要性与冗余性。
Neural Network Pruning via QUBO Optimization
- 融合梯度敏感度与激活相似性构建联合优化目标
- 在SIDD数据集上剪枝后性能优于传统方法2.1%以上
- 适合需要可解释性压缩的模型部署场景
神经网络剪枝可建模为组合优化问题,但现有方法多依赖忽略滤波器间复杂交互的贪心启发式。形式化优化方法如无约束二次二值优化(QUBO)虽具理论优势,却因基于L1范数等简化目标函数而表现欠佳。本文提出统一的混合QUBO框架,将一阶泰勒展开与二阶费舍尔信息等梯度感知敏感度指标引入线性项,以数据驱动的激活相似性作为二次项,使目标函数同时捕捉单个滤波器的重要性与滤波器间的功能冗余性。进一步设计动态容量驱动搜索,严格满足目标稀疏度且不扭曲优化空间。最后采用两阶段流程:先由QUBO求解,再通过张量训练(TT)精修阶段——一种无需梯度的优化器,在真实评估指标上直接微调解。在SIDD图像去噪数据集上的实验表明,该混合QUBO显著优于贪心泰勒剪枝和传统L1-QUBO,TT精修在合适组合尺度下带来持续提升。结果凸显了混合组合形式在鲁棒、可扩展、可解释的神经网络压缩中的潜力。
原文摘要 · Abstract (English)
Neural network pruning can be formulated as a combinatorial optimization problem, yet most existing approaches rely on greedy heuristics that ignore complex interactions between filters. Formal optimization methods such as Quadratic Unconstrained Binary Optimization (QUBO) provide a principled alternative but have so far underperformed due to oversimplified objective formulations based on metrics like the L1-norm. In this work, we propose a unified Hybrid QUBO framework that bridges heuristic importance estimation with global combinatorial optimization. Our formulation integrates gradient-aware sensitivity metrics - specifically first-order Taylor and second-order Fisher information - into the linear term, while utilizing data-driven activation similarity in the quadratic term. This allows the QUBO objective to jointly capture individual filter relevance and inter-filter functional redundancy. We further introduce a dynamic capacity-driven search to strictly enforce target sparsity without distorting the optimization landscape. Finally, we employ a two-stage pipeline featuring a Tensor-Train (TT) Refinement stage - a gradient-free optimizer that fine-tunes the QUBO-derived solution directly against the true evaluation metric. Experiments on the SIDD image denoising dataset demonstrate that the proposed Hybrid QUBO significantly outperforms both greedy Taylor pruning and traditional L1-based QUBO, with TT Refinement providing further consistent gains at appropriate combinatorial scales. This highlights the potential of hybrid combinatorial formulations for robust, scalable, and interpretable neural network compression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。