将大模型拆分到多台服务器运行，降低对高端硬件的依赖。
Multi Part Deployment of Neural Network
- 把神经网络按神经元分片部署在多个服务器上，通过元数据动态调用远程计算。
- 支持灵活分区策略，可在不同层或比例下拆分模型，实现高效分布式训练。
- 适合大规模模型部署，特别适合资源有限但需运行超大模型的团队。
现代神经网络规模持续扩大，如IBM模型达5300亿神经元，Google模型达5000亿参数，带来巨大计算成本与基础设施压力。传统依赖单一GPU集群的训练方式已难以为继。本文提出一种分布式架构，将神经网络拆分至多台服务器，每台负责部分神经元。神经元分为本地与远程，跨服务器通信通过元数据驱动的查找机制管理。多部分神经网络执行引擎通过存储的元数据动态解析并调用远程神经元，实现跨分区无缝执行与训练。所有服务器通过网络文件系统（NFS）共享统一模型，确保并行更新时的一致性。神经元分配模块支持基于神经元数量、占比、标识符或网络层的灵活分区策略。该架构使深度学习模型可在云环境中低成本、可扩展地部署，减少对高性能集中式计算资源的依赖。
原文摘要 · Abstract (English)
The increasing scale of modern neural networks, exemplified by architectures from IBM (530 billion neurons) and Google (500 billion parameters), presents significant challenges in terms of computational cost and infrastructure requirements. As deep neural networks continue to grow, traditional training paradigms relying on monolithic GPU clusters become increasingly unsustainable. This paper proposes a distributed system architecture that partitions a neural network across multiple servers, each responsible for a subset of neurons. Neurons are classified as local or remote, with inter-server connections managed via a metadata-driven lookup mechanism. A Multi-Part Neural Network Execution Engine facilitates seamless execution and training across distributed partitions by dynamically resolving and invoking remote neurons using stored metadata. All servers share a unified model through a network file system (NFS), ensuring consistency during parallel updates. A Neuron Distributor module enables flexible partitioning strategies based on neuron count, percentage, identifiers, or network layers. This architecture enables cost-effective, scalable deployment of deep learning models on cloud infrastructure, reducing dependency on high-performance centralized compute resources.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。