解决分解大模型并行推理效率低的问题，提升大规模模型部署性能。
DeInfer: Efficient Parallel Inferencing for Decomposed Large Language Models
- 通过多级优化设计，提升分解后大模型的并行推理效率。
- 实验表明，相较现有方法，推理速度显著提升，支持更大规模模型扩展。
- 兼容主流优化技术，适合需要高效推理的工业级大模型应用。
现有大语言模型（LLM）分解工作主要关注下游任务性能提升，但忽视了模型规模扩大时并行推理性能不佳的问题。为此，本文提出 DeInfer，一个专为分解大模型设计的高性能并行推理系统。该系统包含多项优化策略，最大化推理性能，并与当前最先进的优化技术兼容。大量实验评估显示，DeInfer 在多个场景下表现优越，证明其能有效促进分解大模型的并行推理能力。
原文摘要 · Abstract (English)
Existing works on large language model (LLM) decomposition mainly focus on improving performance on downstream tasks, but they ignore the poor parallel inference performance when trying to scale up the model size. To mitigate this important performance issue, this paper introduces DeInfer, a high-performance inference system dedicated to parallel inference of decomposed LLMs. It consists of multiple optimizations to maximize performance and be compatible with state-of-the-art optimization techniques. Extensive experiments are carried out to evaluate DeInfer's performance, where the results demonstrate its superiority, suggesting it can greatly facilitate the parallel inference of decomposed LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。