通过激活空间低秩变换，实现大模型无训练压缩与快速部署。
FLAT-LLM: Fine-grained Low-rank Activation Space Transformation for Large Language Model Compression
- 在激活空间用主成分分析计算权重变换，精细控制压缩粒度。
- 无需微调，5个模型11个数据集上表现优于结构剪枝方法。
- 压缩后推理更快，校准仅需几分钟，适合边缘设备部署。
大型语言模型在自然语言处理中取得显著进展，但其高计算和内存需求限制了在资源受限环境中的部署。尽管近期的低秩分解方法为结构压缩提供了前景，但常伴随准确率下降、校准成本高及模型架构效率低等问题。本文提出FLAT-LLM，一种无需训练、快速准确的结构压缩方法，基于激活空间的细粒度低秩变换。具体而言，通过头级主成分分析计算截断特征向量，降低隐藏维度，并采用贪婪预算重分配策略，自适应地在解码器间分配秩。该方法在不进行恢复微调的情况下实现高效权重压缩，校准可在数分钟内完成。在5个模型和11个数据集上的评估显示，FLAT-LLM在泛化能力和下游任务表现上优于结构剪枝基线，同时相比基于分解的方法实现了更快的推理速度。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have enabled remarkable progress in natural language processing, yet their high computational and memory demands pose challenges for deployment in resource-constrained environments. Although recent low-rank decomposition methods offer a promising path for structural compression, they often suffer from accuracy degradation, expensive calibration procedures, and result in inefficient model architectures that hinder real-world inference speedups. In this paper, we propose FLAT-LLM, a fast and accurate, training-free structural compression method based on fine-grained low-rank transformations in the activation space. Specifically, we reduce the hidden dimension by transforming the weights using truncated eigenvectors computed via head-wise Principal Component Analysis, and employ a greedy budget redistribution strategy to adaptively allocate ranks across decoders. FLAT-LLM achieves efficient and effective weight compression without recovery fine-tuning, which could complete the calibration within a few minutes. Evaluated across 5 models and 11 datasets, FLAT-LLM outperforms structural pruning baselines in generalization and downstream performance, while delivering inference speedups over decomposition-based methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。