不依赖校准数据，通过数值与结构双重敏感性实现分层混合精度量化
Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivity
- 从数值和结构两方面分解层内模块，评估其敏感性
- 无需校准数据，在多种模型上均超越基线性能
- 适合对推理效率要求高且无可用校准数据的场景
分层混合精度量化（LMPQ）在极低比特设置下可通过为敏感层分配更高精度实现有效压缩。然而，现有方法通常对层内权重模块一视同仁，并仅依赖单一数值属性估算敏感性，忽略了它们不同的操作角色和结构特征。为此，我们提出NSDS，一种基于数值与结构双重敏感性的校准无关LMPQ框架。具体而言，该方法首先将每层机械分解为不同操作角色，并从数值和结构两个角度量化其敏感性；再通过基于MAD-Sigmoid和Soft-OR的鲁棒聚合方案，将双维度得分整合为统一的层级度量，指导比特分配。大量实验表明，NSDS在多种模型和下游任务中均显著优于各类基线，且无需任何校准数据。
原文摘要 · Abstract (English)
Layer-wise mixed-precision quantization (LMPQ) enables effective compression under extreme low-bit settings by allocating higher precision to sensitive layers. However, existing methods typically treat all intra-layer weight modules uniformly and rely on a single numerical property when estimating sensitivity, overlooking their distinct operational roles and structural characteristics. To address this, we propose NSDS, a novel calibration-free LMPQ framework driven by Numerical and Structural Dual-Sensitivity. Specifically, it first mechanistically decomposes each layer into distinct operational roles and quantifies their sensitivity from both numerical and structural perspectives. These dual-aspect scores are then aggregated into a unified layer-wise metric through a robust aggregation scheme based on MAD-Sigmoid and Soft-OR to guide bit allocation. Extensive experiments demonstrate that NSDS consistently achieves superior performance compared to various baselines across diverse models and downstream tasks, without relying on any calibration data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。