通过反复优化块间边界,提升低比特量化模型的精度。
From Sweep to Seam: Interleaved Cross-Block Post-Training Quantization
- 交替重访相邻块的连接处,两次优化边界信息。
- 在3比特和2比特量化下,显著降低困惑度,避免精度崩溃。
- 无需额外校准数据,兼容现有量化流程,适合部署优化。
将大语言模型压缩至每参数2比特或更少,通过分块后训练量化已变得可行;跨块方法在滑动窗口内重构相邻Transformer块。本文研究固定两块设置下的序列基线,其仅遍历网络一次,早期引入的误差无法修正。提出交错跨块量化(ICBQ),通过反复访问连续块间的边界对进行优化,每个边界对在前一块末尾与后一块开头各被修正一次。该方法保持局部双块目标,复用现有分块后训练量化流水线的校准输入。在局部收缩与平滑性假设下,推导出深度相关的上界:边界重访使传播项放大,但残差项与深度无关。实验表明,相较于匹配的序列交叉块量化基线,ICBQ在三值量化中降低困惑度,在基线严重退化时仍保持有限困惑度,并可扩展至3比特与2比特GPTQ。
原文摘要 · Abstract (English)
Compressing large language models to two bits or fewer is increasingly feasible through block-wise post-training quantization; cross-block variants reconstruct neighboring Transformer blocks within a moving window. In the fixed two-block setting studied here, the matched sequential baseline moves this window through the network once, so errors introduced early in the sweep are not revisited. We propose Interleaved Cross-Block Quantization (ICBQ), a scheduling modification that revisits the boundary pair between consecutive chunks. Each seam pair is refined twice: first at the end of one chunk and again at the start of the next. The method retains the local two-block objective and reuses the calibration inputs of existing block-wise PTQ pipelines. Under stated local contraction and smoothness assumptions, we derive a depth-wise upper-bound comparison in which seam revisits multiply the propagated term while the residual remains bounded independently of depth. In the reported experiments, ICBQ reduces ternary-quantization perplexity relative to the matched Sequential CBQ baseline, yields finite perplexity in configurations where the baseline has severe degradation, and can also be used with 3-bit and 2-bit GPTQ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。