统一评估大模型压缩的三类方法，发现性能与安全不一致。
UniComp: A Unified Evaluation of Large Language Model Compression via Pruning, Quantization, and Distillation
- 构建统一框架，综合评估剪枝、量化、蒸馏三类压缩技术。
- 压缩后事实记忆保留，但推理、多语言等能力明显下降。
- 针对任务校准可提升50%推理性能，适合部署优化场景。
模型压缩对大语言模型（LLM）部署至关重要，但现有对比研究多聚焦剪枝与量化，且主要基于知识导向基准。为此，我们提出UniComp，一个统一评估剪枝、量化与知识蒸馏的框架。该框架从性能、可靠性与效率三个维度，结合多样化的能力建模与安全导向基准，以及硬件感知的效率分析，对七种压缩技术在40多个数据集上进行评估。结果发现：(i) 存在持续的知识偏差，事实回忆能力基本保留，但多步推理、多语言及指令遵循能力显著退化；(ii) 性能与可靠性解耦，保留的性能不代表安全、公平与隐私仍被保持；(iii) 针对任务进行校准可使剪枝模型的推理性能提升最高达50%相对改善。
原文摘要 · Abstract (English)
Model compression is increasingly essential for deploying large language models (LLMs), yet existing comparative studies largely focus on pruning and quantization evaluated primarily on knowledge-centric benchmarks. Thus, we introduce UniComp, a unified evaluation framework for comparing pruning, quantization, and knowledge distillation. UniComp evaluates compressed models along three dimensions: performance, reliability, and efficiency, using a diverse set of capability- and safety-oriented benchmarks together with a hardware-aware efficiency analysis. Through evaluation of seven compression techniques across over 40 datasets, we observe (i) a consistent knowledge bias, where factual recall is largely preserved while multi-step reasoning, multilingual, and instruction-following capabilities degrade; (ii) a deployment-critical performance-reliability decoupling, where retained performance does not indicate preserved safety, fairness and privacy; and (iii) that task-specific calibration can yield up to 50% relative improvement in reasoning performance in pruned models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。