通过重构计算流程，大幅减少多GPU推理时的同步开销。
Parallel Track Transformers: Enabling Fast GPU Inference with Reduced Synchronization
- 将Transformer计算拆分为独立并行轨道，降低跨GPU依赖
- 同步操作减少16倍，首词生成延迟降低15%-30%
- 适用于主流LLM部署框架，提升整体服务吞吐量
大规模Transformer语言模型的高效推理仍是系统核心挑战，通常需多GPU并行以满足低延迟和高吞吐要求。传统张量并行虽将矩阵运算分布到多个设备，但引入大量跨GPU同步，导致通信瓶颈，降低可扩展性。本文提出并行轨道（Parallel Track, PT）Transformer，一种新型架构范式，通过重构计算以最小化跨设备依赖。实验表明，相比标准张量并行，PT可减少高达16倍的同步操作，同时保持竞争力的模型质量。我们将PT集成至两个主流LLM服务栈——TensorRT-LLM和vLLM，均实现显著性能提升：首词生成时间降低15%-30%，每输出词耗时减少2%-12%，吞吐量最高提升31.90%。
原文摘要 · Abstract (English)
Efficient large-scale inference of transformer-based large language models (LLMs) remains a fundamental systems challenge, frequently requiring multi-GPU parallelism to meet stringent latency and throughput targets. Conventional tensor parallelism decomposes matrix operations across devices but introduces substantial inter-GPU synchronization, leading to communication bottlenecks and degraded scalability. We propose the Parallel Track (PT) Transformer, a novel architectural paradigm that restructures computation to minimize cross-device dependencies. PT achieves up to a 16x reduction in synchronization operations relative to standard tensor parallelism, while maintaining competitive model quality in our experiments. We integrate PT into two widely adopted LLM serving stacks-Tensor-RT-LLM and vLLM-and report consistent improvements in serving efficiency, including up to 15-30% reduced time to first token, 2-12% reduced time per output token, and up to 31.90% increased throughput in both settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。