arXiv:2512.12981cs.LGstat.ML2025-12被引 2

用可学习的死区量化实现端到端压缩,无需额外调参。

CoDeQ: End-to-End Joint Model Compression with Dead-Zone Quantizer for High-Sparsity and Low-Precision Networks

  • 通过可训练死区量化直接在量化层引入稀疏性,实现剪枝与量化联合优化。
  • ResNet-18在ImageNet上压缩至5%比特运算量,精度接近全精度模型。
  • 无需额外搜索过程,支持固定与混合精度,适合部署场景快速适配。

尽管联合剪枝与量化理论上优于顺序执行,但现有方法依赖训练循环外的辅助流程来确定压缩参数,增加了工程复杂度和超参数调优负担,且缺乏直接的数据驱动梯度信号,可能导致次优压缩。本文提出CoDeQ,一种简单、完全可微的联合剪枝-量化方法。核心思想是:标量量化器的死区等价于幅度剪枝,可直接在量化算子中诱导稀疏性。具体地,我们参数化死区宽度,并通过反向传播学习该参数,同时优化量化参数。该设计实现了对稀疏性的显式控制,由单一全局超参数正则化,且将稀疏性选择与位宽选择解耦。结果是,CoDeQ支持固定精度和混合精度量化(通过可选的第二个超参数控制),可在单一端到端优化中同时确定稀疏模式与量化参数。因此,CoDeQ无需任何辅助流程,具有架构无关性,实现简便。在ImageNet上的ResNet-18实验表明,CoDeQ将比特操作降至约5%,在固定精度和混合精度设置下均保持接近全精度的准确率。

原文摘要 · Abstract (English)

While joint pruning--quantization is theoretically superior to sequential application, current joint methods rely on auxiliary procedures outside the training loop for finding compression parameters. This reliance adds engineering complexity and hyperparameter tuning, while also lacking a direct data-driven gradient signal, which might result in sub-optimal compression. In this paper, we introduce CoDeQ, a simple, fully differentiable method for joint pruning--quantization. Our approach builds on a key observation: the dead-zone of a scalar quantizer is equivalent to magnitude pruning, and can be used to induce sparsity directly within the quantization operator. Concretely, we parameterize the dead-zone width and learn it via backpropagation, alongside the quantization parameters. This design provides explicit control of sparsity, regularized by a single global hyperparameter, while decoupling sparsity selection from bit-width selection. The result is a method for Compression with Dead-zone Quantizer (CoDeQ) that supports both fixed-precision and mixed-precision quantization (controlled by an optional second hyperparameter). It simultaneously determines the sparsity pattern and quantization parameters in a single end-to-end optimization. Consequently, CoDeQ does not require any auxiliary procedures, making the method architecture-agnostic and straightforward to implement. On ImageNet with ResNet-18, CoDeQ reduces bit operations to ~5% while maintaining close to full precision accuracy in both fixed and mixed-precision regimes.

模型压缩量化剪枝端到端

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。