通过逐层裁剪让大模型在特定领域更快，不依赖硬件也能提速。
TrimLLM: Progressive Layer Dropping for Domain-Specific LLMs
- 基于层级特化现象，逐步裁掉低效模型层。
- 在50%~60%压缩率下保持准确率，速度提升2.1~5.7倍。
- 适合需要低延迟、隐私保护的医疗、法律等场景部署。
为满足本地部署中对性能、延迟和隐私的要求，将大语言模型（LLM）适配到特定领域至关重要。然而，传统任务定制方法无法同时实现内存节省和推理加速。现有压缩技术如量化和剪枝需专用硬件或内核支持才能获得实际加速。我们基于在现代LLM上观察到的层级特化现象，提出TrimLLM，通过渐进式层裁剪减少模型深度。该方法在特定领域保留模型能力，并在不依赖硬件与深度学习框架的前提下实现推理加速。我们在多种规模的LLM上评估了TrimLLM，针对医疗、法律和金融数据集训练的模型，在消费级GPU上实现2.1–5.7倍的推理加速，A100上最高达3.1倍，相比当前最优压缩算法，在50%~60%模型压缩率下无准确率损失。
原文摘要 · Abstract (English)
Specializing large language models (LLMs) for local deployment in domain-specific use cases is necessary for strong performance while meeting latency and privacy constraints. However, conventional task-specific adaptation approaches do not show simultaneous memory saving and inference speedup at deployment time. Practical compression techniques like quantization and pruning require dedicated hardware or kernel support to achieve measured inference speedup. We develop TrimLLM based on the layer-wise specialization phenomenon we empirically observed and verified on contemporary LLMs. TrimLLM reduces the depth of LLMs via progressive layer dropping. We show it retains LLMs' capacity in specific domains and achieves inference speedup irrespective of hardware and deep learning frameworks. We evaluated TrimLLM on LLMs of various sizes for inference; models adapted on medical, legal, and financial datasets all demonstrate $2.1-5.7\times$ inference speedup on consumer GPUs and up to $3.1\times$ speedup on A100 when compared to state-of-the-art model compression algorithms, with no loss in accuracy at 50$\sim$60\% model compression ratio.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。