同时优化模型剪枝与混合精度量化,显著提升低比特大模型的推理质量。
Joint Structural Pruning and Mixed-Precision Quantization for LLM Compression

- 端到端联合优化剪枝与混合精度量化策略,全局控制误差传播。
- 1-3比特下,相比最优基线,WikiText困惑度降低21%。
- 适合追求极致压缩比与高推理精度的工业级大模型部署场景。
大语言模型(LLM)的部署效率已成为实际应用中的关键问题。尽管训练后量化(PTQ)和结构化剪枝是降低内存占用与推理延迟的成熟技术,但现有大部分PTQ方法仅在单层层面优化量化误差,忽略了误差在网络中的累积与传播,常导致次优结果。传统流程也多将剪枝与量化分离或顺序执行,进一步加剧了次优性。本文提出一种新颖的端到端框架,从两方面突破:首先,设计一种新型混合精度PTQ策略,直接最小化全模型范围内的全局误差传播,而非孤立优化各层误差;其次,构建联合优化方法,在统一搜索空间中同步学习结构剪枝决策与混合精度量化策略。大量实验表明,在超低精度(1-3比特)条件下,本方法相较于最先进的权重-激活量化基线,可使WikiText困惑度降低高达21%;相比领先权重量化方法,在WikiText和C4数据集上分别实现59%和85%的更低困惑度。与当前最优的剪枝-量化联合技术相比,本方法在极低比特下仍能取得更优的困惑度与推理性能。
原文摘要 · Abstract (English)
Recently, the efficiency of Large Language Models (LLMs) deployment has become a critical concern in practical applications. While post-training quantization (PTQ) and structural pruning are established techniques for reducing memory footprint and inference latency, most existing PTQ approaches optimize quantization errors on a per-layer basis, overlooking how errors accumulate and propagate through the network, often resulting in suboptimal solutions. Traditional pipelines also tend to apply pruning and quantization in isolation or sequentially, further compounding sub-optimality. We introduce a novel end-to-end framework that addresses these limitations in two key ways. First, we propose a novel mixed-precision PTQ strategy that directly minimizes global error propagation across the entire model, rather than isolating layer-wise errors. Building on this, we develop a novel joint optimization approach that simultaneously learns structural pruning decisions and mixed-precision quantization policies within a unified search space. Extensive experiments show that, at ultra-low precisions (1-3 bits), our quantization method reduces WikiText perplexity by up to 21% compared to state-of-the-art (SoTA) weight-activation quantization baselines. Against leading weight-only quantization methods, it achieves up to 59% and 85% lower perplexity on WikiText and C4, respectively. Compared to the SoTA joint pruning-and-quantization techniques, our proposed method delivers superior perplexity and reasoning performance at ultra-low bits.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。