4-bit量化对低资源语言翻译质量影响大，2比特更差。
The Uneven Impact of Post-Training Quantization in Machine Translation
- 对比四种量化方法，发现模型大小与算法共同决定鲁棒性。
- 2比特时低资源和语系多样语言翻译性能显著下降。
- 用目标语言校准能提升低比特场景下的翻译效果。
量化是将大语言模型部署到资源受限硬件的关键技术，但其在多语言任务中的影响仍不明确。本文首次在55种语言上对5个参数量从1.7B到70B的大语言模型进行大规模后训练量化（PTQ）评估。结果表明，4比特量化通常可保持高资源语言的翻译质量，但对低资源语言和语系多样性语言会产生明显退化，尤其在2比特设置下更为严重。我们对比了四种量化方法（AWQ、BitsAndBytes、GGUF、AutoRound），发现算法选择与模型规模共同决定鲁棒性，其中GGUF变体在2比特精度下表现最稳定。此外，我们量化分析了量化、解码超参数与校准语言之间的交互作用，发现语言匹配的校准主要在低比特场景中带来优势。研究为在量化约束下部署多语言机器翻译系统提供了可操作建议，尤其适用于低资源场景。
原文摘要 · Abstract (English)
Quantization is essential for deploying large language models (LLMs) on resource-constrained hardware, but its implications for multilingual tasks remain underexplored. We conduct the first large-scale evaluation of post-training quantization (PTQ) on machine translation across 55 languages using five LLMs ranging from 1.7B to 70B parameters. Our analysis reveals that while 4-bit quantization often preserves translation quality for high-resource languages and large models, significant degradation occurs for low-resource and typologically diverse languages, particularly in 2-bit settings. We compare four quantization techniques (AWQ, BitsAndBytes, GGUF, and AutoRound), showing that algorithm choice and model size jointly determine robustness. GGUF variants provide the most consistent performance, even at 2-bit precision. Additionally, we quantify the interactions between quantization, decoding hyperparameters, and calibration languages, finding that language-matched calibration offers benefits primarily in low-bit scenarios. Our findings offer actionable insights for deploying multilingual LLMs for machine translation under quantization constraints, especially in low-resource settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。