arXiv:2601.18306cs.CLcs.AI2026-01Conference of the …被引 1

多语言校准能显著提升量化大模型的性能,尤其对非英语更有效。

Calibrating Beyond English: Language Diversity for Better Quantized Multilingual LLM

  • 用多语言数据做量化校准,比仅用英语效果更好。
  • 在10种语言上测试,多语言组合使困惑度降低最多3.52点。
  • 针对不同语言定制校准数据,可避免性能下降,适合多语种部署。

量化是降低大语言模型存储与计算开销的有效方法,但常导致性能下降。现有后训练量化方法多使用小规模纯英文校准集,其对多语言模型的影响尚未充分研究。我们系统评估了在两个量化器(GPTQ、AWQ)上,针对10种语言采用八种校准设置(五种单语言和三种多语言混合)的效果。结果表明:非英语及多语言校准集显著优于纯英文基线,在Llama3.1 8B和Qwen2.5 7B上均实现平均困惑度提升,多语言混合方案最高使困惑度降低3.52点。分析还发现,按目标语言定制校准集可带来最大收益,凸显语言对齐的重要性。此外,部分语言-量化器组合出现性能退化,源于不同语言间激活分布差异。这说明静态统一校准不理想,校准数据的语言多样性与针对性至关重要。

原文摘要 · Abstract (English)

Quantization is an effective technique for reducing the storage footprint and computational costs of Large Language Models (LLMs), but it often results in performance degradation. Existing post-training quantization methods typically use small, English-only calibration sets; however, their impact on multilingual models remains underexplored. We systematically evaluate eight calibration settings (five single-language and three multilingual mixes) on two quantizers (GPTQ, AWQ) on data from 10 languages. Our findings reveal a consistent trend: non-English and multilingual calibration sets significantly improve perplexity compared to English-only baselines. Specifically, we observe notable average perplexity gains across both quantizers on Llama3.1 8B and Qwen2.5 7B, with multilingual mixes achieving the largest overall reductions of up to 3.52 points in perplexity. Furthermore, our analysis indicates that tailoring calibration sets to the evaluation language yields the largest improvements for individual languages, underscoring the importance of linguistic alignment. We also identify specific failure cases where certain language-quantizer combinations degrade performance, which we trace to differences in activation range distributions across languages. These results highlight that static one-size-fits-all calibration is suboptimal and that tailoring calibration data, both in language and diversity, plays a crucial role in robustly quantizing multilingual LLMs.

量化多语言大模型校准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。