arXiv:2608.24615cs.CL2026-08

首次系统评估量化对孟加拉语大模型理解能力的影响,发现模型选择比量化位数更重要。

Quantization Effects on Bangla Language Understanding in Large Language Models: A Systematic Evaluation

论文配图:Quantization Effects on Bangla Language Understanding in Large Language Models: A Systematic Evaluation
图 1 · 摘自论文原文
  • 对比三种模型在五项孟加拉语任务上全精度与三种量化格式的表现
  • 部分模型量化后准确率下降超57%,但少数情况下反而提升
  • 提示开发者选型时需关注架构与量化方法匹配度

后训练量化可降低大语言模型的内存占用并加速推理,适用于设备端部署。然而,现有研究多基于英语基准,对语法复杂、资源稀缺的孟加拉语尚不明确。本文首次在五个孟加拉语自然语言理解基准(Bangla MMLU、CommonsenseQA-BN、OpenBookQA-BN、PIQA-BN、BoolQ-BN)上,通过零样本评估框架lm-evaluation-harness,系统比较了Qwen-2.5-7B、LLaMA-3.1-8B和GPT-OSS-20B三个模型家族在全精度及GPTQ-Int8、GPTQ-Q8、GGUF-W8A16三种量化格式下的表现。结果表明:不同模型响应差异显著——GPT-OSS在GGUF-W8A16下推理类任务最高损失达57.35%准确率;而Qwen与LLaMA在GPTQ格式下保持稳定,部分任务甚至优于全精度版本;BoolQ-BN(理解类任务)在所有模型与格式中均保持稳定。整体表明量化可用于孟加拉语部署,但模型架构与量化方法的选择至关重要,仅依赖位宽不足以保证效果。

原文摘要 · Abstract (English)

Post-training quantization lowers the memory footprint of Large Language Models (LLMs) and speeds up inference, which is why it is now common for on-device deployment. Most of what we know about its effects, however, comes from English benchmarks. It is not clear whether the same holds for morphologically complex, low-resource languages such as Bangla, and this gap is what we address here. We evaluate three model families---Qwen-2.5-7B, LLaMA-3.1-8B, and GPT-OSS-20B---in full precision and in three quantized formats (GPTQ-Int8, GPTQ-Q8, GGUF-W8A16) across five Bangla natural language understanding benchmarks (Bangla MMLU, CommonsenseQA-BN, OpenBookQA-BN, PIQA-BN, and BoolQ-BN), using zero-shot evaluation through lm-evaluation-harness. To our knowledge this is the first controlled comparison of quantization formats on Bangla NLU. The three families do not respond the same way: GPT-OSS loses up to 57.35% accuracy on reasoning-heavy tasks under GGUF-W8A16, while Qwen and LLaMA hold steady under GPTQ, and in a few cases the quantized version edges out the full-precision one. BoolQ-BN, a comprehension task, stays stable across all three families regardless of format. Taken together, these results suggest quantization can work well for Bangla deployment, but the choice of architecture and quantization method matters more than the bit width alone. We discuss what this means for practitioners choosing a model to run on constrained hardware.

语言理解量化低资源语言大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。