通过细粒度核级拆分,提升异构GPU集群的推理性能与成本效益。
Tessera: Unlocking Heterogeneous GPUs through Kernel-Granularity Disaggregation

- 以计算核为单位拆分任务,精准匹配不同GPU硬件特性。
- 在16张GPU上实现最高2.3倍吞吐提升,成本效率提升1.6倍。
- 适用于多种模型架构,尤其适合异构硬件部署场景。
拆分技术将AI工作负载的部分映射到不同类型的GPU上,为利用现代异构GPU集群提供路径。然而,现有方案仅在粗粒度层面操作,且紧密耦合于特定模型架构,仍有巨大性能优化空间。本文提出Tessera,首个面向大规模模型推理的核粒度拆分系统,可显著提升异构GPU的性能与成本效率。核心洞察是:单个应用内的计算核具有差异化的资源需求,是最佳对齐计算与硬件能力的粒度。Tessera结合离线分析与在线适应,从PTX中提取精确的核间依赖关系以保证正确性,通过流水线执行模型重叠通信与计算,并采用感知负载的调度策略及轻量级运行时自适应。在五种异构GPU和四种模型架构上进行的广泛评估显示,相较于现有拆分方法,Tessera在扩展至16张GPU时,服务吞吐提升最高达2.3倍,成本效率提升1.6倍,且能推广至先前方法无法适用的模型架构。令人惊讶的是,一组异构GPU在Tessera下甚至能超越两台同质高端GPU的吞吐表现,且成本更低。
原文摘要 · Abstract (English)
Disaggregation maps parts of an AI workload to different types of GPUs, offering a path to utilize modern heterogeneous GPU clusters. However, existing solutions operate at a coarse granularity and are tightly coupled to specific model architectures, leaving much room for performance improvement. This paper presents Tessera, the first kernel disaggregation system to improve performance and cost efficiency on heterogeneous GPUs for large model inference. Our key insight is that kernels within a single application exhibit diverse resource demands, making them the most suitable granularity for aligning computation with hardware capabilities. Tessera integrates offline analysis with online adaptation by extracting precise inter-kernel dependencies from PTX to ensure correctness, overlapping communication with computation through a pipelined execution model, and employing workload-aware scheduling with lightweight runtime adaptation. Extensive evaluations across five heterogeneous GPUs and four model architectures, scaling up to 16 GPUs, show that Tessera improves serving throughput and cost efficiency by up to 2.3x and 1.6x, respectively, compared to existing disaggregation methods, while generalizing to model architectures where prior approaches do not apply. Surprisingly, a heterogeneous GPU pair under Tessera can even exceed the throughput of two homogeneous high-end GPUs at a lower cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。