针对量化导致非英语语言性能下降的问题，提出轻量级修复方法LCD。
Language-Conditional Dequantization: Recovering What Quantization Steals from Non-English Languages
- 为量化模型添加按语言定制的微调模块，仅增0.12%参数
- 恢复非拉丁语系70-83%的困惑度损失，提升全球测评准确率17-28%
- 特别适合多语言场景，对语言差异大的任务效果显著
激进量化会严重损害多语言能力：在子4B INT3 GPTQ设置下，非英语语言的困惑度退化比英语高2-4倍。我们提出语言条件去量化（Language-Conditional Dequantization, LCD），一种后处理方法，在已量化模型的线性层上附加每语言的秩2 LoRA修正模块，每语言仅增加0.12%参数，单卡训练不到20分钟。在Qwen2.5-3B和Llama-3.2-3B上，LCD恢复了非拉丁文字语言70-83%的困惑度差距，以及全球MMLU准确率17-28%的差距。相比同容量无语言区分修正，其在语言类型差异大的任务上高出3-9点；相比数据无关低秩基线（LQER），性能提升一个数量级。我们进一步发现困惑度与准确率间的脱节现象，并定位到量化损伤集中区域：早期深度错误（如Llama）会向下游传播且难以局部修正，而晚期深度错误（如Qwen）则不会。通过限制层的LCD变体直接验证了该机制。
原文摘要 · Abstract (English)
Aggressive quantization disproportionately harms multilingual capability: in the sub-4B INT3 GPTQ regime, we measure 2-4x larger perplexity degradation on non-English languages than on English. We propose Language-Conditional Dequantization (LCD), a post-hoc method that attaches per-language rank-2 LoRA corrections to the linear layers of an already-quantized model, adding 0.12% parameters per language and training in under 20 minutes on a single GPU. Across Qwen2.5-3B and Llama-3.2-3B, LCD recovers 70-83% of the perplexity gap for non-Latin script languages and 17-28% of the GlobalMMLU accuracy gap, outperforming a language-agnostic correction of equal capacity by 3-9 points on typologically distant languages and a data-free low-rank baseline (LQER) by an order of magnitude. We further identify a perplexity-accuracy disconnect and trace it to where quantization concentrates damage: early-depth errors (Llama) propagate downstream and resist local correction, while late-depth errors (Qwen) do not. A layer-restricted variant of LCD validates this mechanism directly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。