量化压缩模型时兼顾可信度，提升高风险场景可靠性
Enhancing Trustworthiness with Mixed Precision: Benchmarks, Opportunities, and Challenges
- 用混合精度投票方法融合不同精度版本的模型预测
- 在四项可信度指标上最高提升5.8%表现
- 适合医疗金融等对安全要求高的应用研究
大语言模型在各类任务中表现优异，但自回归解码过程对现有AI硬件部署带来效率挑战。量化通过将权重、激活值和键值缓存压缩至低精度，在保持生成质量的同时缓解内存与计算压力。然而，现有量化框架多关注困惑度或分类准确率，常忽略关键可信度指标。这一差距在金融、医疗等高风险下游场景中引入潜在风险。本文系统研究量化对四类可信度指标（对抗鲁棒性、公平性、机器伦理、分布外鲁棒性）的影响，发现其性能随压缩比和量化方法变化不稳定。基于此，我们提出一种新颖的精度集成投票方法，利用同一模型的混合精度变体进行预测，使可信度指标性能最高提升5.8%。结果表明，在开发模型压缩技术时必须考虑可信度，为压缩与可信度交叉研究开辟新方向，尤其适用于安全关键应用。
原文摘要 · Abstract (English)
Large language models (LLMs) have shown promising performance across various tasks. However, their autoregressive decoding process poses significant challenges for efficient deployment on existing AI hardware. Quantization alleviates memory and compute pressure by compressing weights, activations, and KV caches to low precisions while preserving generation quality. However, existing quantization frameworks typically focus on perplexity or classification accuracy, often omitting critical trustworthiness metrics. This gap introduces risks when applying quantized LLMs to downstream high-stakes domains such as finance and healthcare. In this work, we systematically investigate the impact of quantization on four trustworthiness metrics (adversarial robustness, fairness, machine ethics, and out-of-distribution robustness) and identify the instability across compression ratios and quantization methods. Building on these observations, we develop a novel precision-ensemble voting approach that leverages predictions from mixed-precision variants of the same model and consistently improves performance by up to $5.8\%$ on trustworthiness metrics. Our results highlight the importance of considering trustworthiness when developing model compression techniques and point to research opportunities at the intersection of compression and trustworthiness for safety-critical applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。