提出软边界量化器，提升状态空间模型激活值精度
Late Breaking Results: Quamba-SE: Soft-edge Quantizer for Activations in State Space Models
- 采用三尺度自适应量化，区分小值、常规值和异常值
- 在6个零样本任务上平均准确率提升0.83%，最高达+2.68%
- 适合追求高精度推理的轻量级模型部署场景
我们提出 Quamba-SE，一种用于状态空间模型（SSM）激活值量化的软边界量化器。与现有方法使用标准INT8操作不同，Quamba-SE采用三个自适应尺度：对小值使用高精度，对常规值使用标准尺度，对异常值使用低精度。该方法保留了异常值信息，避免硬截断，同时保持其他值的精度。我们在 Mamba-130M 上评估了6个零样本基准。结果表明，Quamba-SE 始终优于 Quamba，在单个基准上最高提升2.68%，6个数据集平均准确率提升0.83%。
原文摘要 · Abstract (English)
We propose Quamba-SE, a soft-edge quantizer for State Space Model (SSM) activation quantization. Unlike existing methods, using standard INT8 operation, Quamba-SE employs three adaptive scales: high-precision for small values, standard scale for normal values, and low-precision for outliers. This preserves outlier information instead of hard clipping, while maintaining precision for other values. We evaluate on Mamba- 130M across 6 zero-shot benchmarks. Results show that Quamba- SE consistently outperforms Quamba, achieving up to +2.68% on individual benchmarks and up to +0.83% improvement in the average accuracy of 6 datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。