通过并行计算连续层对，提升大模型推理速度
Leveraging the true depth of LLMs
- 将连续层对并行计算，重构模型计算图
- 在Llama 2 7B上实现1.19倍吞吐提升，准确率仅降1.5%
- 无需重训练，适合大规模模型部署
大型语言模型（LLM）虽能力出众，但计算开销巨大。尽管已有研究发现许多模型层可重排甚至移除而对准确率影响甚微，但这些发现尚未带来显著的推理加速。为此，我们提出一种新方法，通过将连续层对分组并行计算来重构计算图，无需重新训练即可在Llama 2 7B上实现1.19倍的吞吐量提升，平均准确率仅下降1.5%。我们验证了该方法在大规模模型部署中的实际价值，并表明通过轻量级微调并行化层，部分损失的准确率可被恢复。
原文摘要 · Abstract (English)
The remarkable capabilities of Large Language Models (LLMs) are overshadowed by their immense computational cost. While recent work has shown that many LLM layers can be reordered or even removed with minimal impact on accuracy, these insights have not been translated into significant inference speedups. To bridge this gap, we introduce a novel method that restructures the computational graph by grouping and evaluating consecutive layer pairs in parallel. This approach, requiring no retraining, yields a 1.19x throughput gain on Llama 2 7B while reducing the average benchmark accuracy by only 1.5\%. We demonstrate the practical value of this method for large-scale LLM deployment and show that some of the lost accuracy can be recovered with lightweight fine-tuning of the parallelized layers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。