arXiv:2605.16882cs.CL2026-05被引 2

通过专家引导与权重锚定,提升合并模型的低比特量化效果。

E-PMQ: Expert-Guided Post-Merge Quantization with Merged-Weight Anchoring

  • 用源专家权重提供校准目标,分层优化合并后模型。
  • 4比特量化下任务算术和TIES合并准确率提升至73.6%和74.8%。
  • 适合需要多任务合并部署的轻量级模型应用。

低资源部署需求使模型量化成为神经网络部署的关键技术。同时,模型合并已成为一种无需联合训练或多模型服务即可整合多个任务或领域专用专家的实用低资源策略。结合量化与模型合并,可实现高效低资源部署:将多个专家集成到一个低比特模型中。本文将此场景定义为后合并量化(PMQ)。我们发现,直接对合并模型应用后训练量化(PTQ)不可靠,因存在两类耦合偏差:由低比特重构引入的量化偏差,以及从模型合并继承的专家相对偏差。为此,我们提出E-PMQ框架,利用源专家权重在逐层校准中提供专家引导输出目标,并结合合并权重锚定稳定校准过程,保留合并模型的集成行为。在CLIP-ViT-B/32八任务合并中,E-PMQ将4比特GPTQ准确率从65.0%提升至73.6%(任务算术),从69.1%提升至74.8%(TIES-Merging)。在更难设置下,20任务CLIP-ViT-L/14上,GPTQ从34.8%提升至76.7%;FLAN-T5-base GLUE上从78.26%提升至83.34%。结果表明,E-PMQ能有效实现后合并量化与低比特部署。

原文摘要 · Abstract (English)

Low-resource deployment constraints have made model quantization essential for deploying neural networks while preserving performance. Meanwhile, model merging has become an increasingly practical low-resource strategy for integrating multiple task- or domain-specialized experts into a single model without joint training or multi-model serving. Together, quantization and model merging enable an efficient low-resource deployment pipeline by integrating multiple experts into one low-bit model. We formulate this setting as Post-Merge Quantization (PMQ). We show that directly applying post-training quantization (PTQ) to a merged model is unreliable because two distinct deviations are coupled: the quantization deviation introduced by low-bit reconstruction and the expert-relative merging deviation inherited from model merging. To mitigate these deviations, we propose E-PMQ, an expert-guided PMQ framework that uses source expert weights to provide expert- guided output targets during layer-wise calibration, together with merged-weight anchoring to stabilize the calibration and preserve the integrated behavior of the merged model. On CLIP-ViT-B/32 eight-task merging, E-PMQ improves 4-bit GPTQ from 65.0% to 73.6% under Task Arithmetic and from 69.1% to 74.8% under TIES-Merging. On harder settings, E-PMQ improves GPTQ from 34.8% to 76.7% on 20-task CLIP-ViT-L/14 and from 78.26% to 83.34% on FLAN-T5- base GLUE. These results demonstrate that E-PMQ enables effective post-merge quantization and low-bit deployment.

模型量化模型合并低比特部署专家引导

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。