提出两种无需训练的压缩优化方法，提升大模型低秩压缩精度
Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs
- 分层压缩并修正校准误差，避免层间激活错位
- 迭代调整各层秩分配，缓解压缩后重要性失真
- 在Llama和Qwen3上实现1-2.5准确率提升，适合部署优化场景
训练无关的低秩压缩框架因其在减少参数量的同时保持任务级准确性而受到关注。然而现有最先进方法存在两大局限：(1) 校准数据激活残差在压缩过程中逐层累积，导致压缩时模拟的表示与推理时实际表现不一致；(2) 假设压缩后各层重要性分布不变，此假设不成立。这两者共同引入压缩过程与部署模型间的表征错位。本文研究该问题，提出一种简单、无需训练的方法，兼容现有框架，包含：(1) 带校准修正的分层压缩；(2) 带秩分配修正的迭代压缩。在Llama和Qwen3模型上，于多个基准测试和压缩率下评估，相比每权重与联合分解基线，在零样本任务上最高提升~1-2.5准确率点。
原文摘要 · Abstract (English)
Training-free low-rank compression frameworks have been gaining prominence for LLM compression given their effectiveness in reducing model parameter count while maintaining task-level accuracy. However, existing SOTA frameworks share two key limitations: (1) residual errors in calibration data activations accumulate across layers during compression, causing misalignment between representations simulated at compression time and those experienced at inference; (2) the assumption that layer importance distribution is preserved post-compression does not hold. Together, these two effects introduce misalignment in the compression process in relation to the deployed model. We study these effects and propose a simple, training-free methodology compatible with existing frameworks to mitigate them, comprising: (1) Layer-by-Layer Compression with Calibration Correction; (2) Iterative Compression with Rank Allocation Correction. Implemented atop an existing SOTA decomposition framework, and evaluated on Llama and Qwen3 models across various benchmarks and compression rates, our approach demonstrates up to ~1-2.5 accuracy point improvements over per-weight and joint decomposition baselines on zero-shot tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。