用苹果M2 Ultra搭建私有大模型，实现高效专家并行推理
Towards Building Private LLMs: Exploring Multi-Node Expert Parallelism on Apple Silicon for Mixture-of-Experts Large Language Model
- 在多台Mac Studio上并行部署MoE模型专家，降低推理延迟
- 专家计算与通信时间相当，网络延迟成关键瓶颈
- 优化内存管理后，成本效率比英伟达H100超算高1.15倍
大型语言模型（LLMs）如OpenAI的ChatGPT、Meta的Llama和Databricks的DBRX推动了人工智能的发展。本文针对个人或小团队构建私有LLM系统面临的成本与可扩展性挑战，利用搭载Apple M2 Ultra芯片的Mac Studio集群，实现了对预训练DBRX模型（采用混合专家MoE架构）的高效部署与加速。性能分析表明，在2至4台机器节点间并行执行专家可显著降低推理时间。我们发现专家计算时间与输出交换的通信时间相当，凸显网络延迟的重要性超过带宽。同时观察到苹果软件栈内存管理逻辑带来显著开销。基于此，我们提出优化方案消除该开销。结果表明，该Mac Studio集群在成本效率上比当前最先进的英伟达H100 GPU超算高出1.15倍。此外，我们构建了性能模型，用于预测不同配置下的系统表现，为私有LLM系统设计提供重要参考。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have revolutionized Artificial Intelligence (AI) with significant advancements such as OpenAI's ChatGPT, Meta's Llama, and Databricks' DBRX. This paper addresses the cost and scalability challenges encountered when constructing private LLM systems for personal or small group services, as aimed by Apple Intelligence. A Mac Studio cluster with Apple's M2 Ultra chips is established as a cost-efficient solution to host and accelerate the pretrained DBRX model with the Mixture-of-Experts (MoE) architecture. Our performance analysis reveal that parallel execution of the model's experts across two to four machine nodes significantly reduces inference time. We find that computation time for the experts is comparable to the communication time for exchanging their outputs, emphasizing the importance of network latency over bandwidth. We also observe significant management overhead due to Apple software stack's memory management logic. Based on these findings, we develop optimization schemes to eliminate the memory management overhead. As a result, the Mac Studio cluster is 1.15 times more cost-efficient than the state-of-the-art AI supercomputer with NVIDIA H100 GPUs. In addition, we construct a performance model to estimate system performance under varying configurations, and the model provides valuable insights for designing private LLM systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。