用元网络将大模型压缩10倍，精度几乎不变。
PocketLLM: Ultimate Compression of Large Language Models via Meta Networks
- 通过编码器将模型权重转为离散隐向量，再用码本压缩。
- 在10倍压缩比下，Llama 2-7B模型精度损失极小。
- 适合需要极致压缩的边缘设备部署场景。
随着大型语言模型（LLMs）规模持续扩大，在边缘设备上存储和传输模型变得愈发困难。传统方法如量化和剪枝难以在不牺牲精度的前提下实现极端压缩。本文提出PocketLLM，一种基于元网络在隐空间压缩LLMs的新方法。通过一个简单编码器将模型权重映射为离散隐向量，并用紧凑码本表示；再通过轻量解码器将码本代表向量还原至原始权重空间。该方法仅需一个小解码器、一个紧凑码本和索引，即可实现对大模型权重的显著压缩。大量实验表明，即使在极高压缩比下，PocketLLM仍保持优异性能，例如将Llama 2-7B压缩10倍，精度下降可忽略不计。
原文摘要 · Abstract (English)
As Large Language Models (LLMs) continue to grow in size, storing and transmitting them on edge devices becomes increasingly challenging. Traditional methods like quantization and pruning struggle to achieve extreme compression of LLMs without sacrificing accuracy. In this paper, we introduce PocketLLM, a novel approach to compress LLMs in a latent space via meta-networks. A simple encoder network is proposed to project the weights of LLMs into discrete latent vectors, which are then represented using a compact codebook. A lightweight decoder network is employed to map the codebook's representative vectors back to the original weight space. This method allows for significant compression of the large weights in LLMs, consisting solely of a small decoder, a concise codebook, and an index. Extensive experiments show that PocketLLM achieves superior performance even at significantly high compression ratios, e.g., compressing Llama 2-7B by 10x with a negligible drop in accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。