arXiv:2604.14552cs.PFcs.AR2026-04被引 1

对比T4与L4 GPU在推理性能上的差距,给出选型建议。

DEEP-GAP: Deep-learning Evaluation of Execution Parallelism in GPU Architectural Performance

论文配图:DEEP-GAP: Deep-learning Evaluation of Execution Parallelism in GPU Architectural Performance
图 1 · 摘自论文原文
  • 用统一配置测试ResNet系列模型,比较不同精度和批次下的表现。
  • L4比T4最高快4.4倍,小批量(16-32)时能效最优。
  • 适合关注延迟和能效的推理部署团队参考选型。

现代数据中心越来越多地采用低功耗、单槽位推理加速器,在性能、能效与机架密度间取得平衡。NVIDIA T4 GPU 因其出色的每瓦性能和成熟的软件支持而广泛应用。其继任者 L4 GPU 在张量核心吞吐量、缓存容量、内存带宽和并行执行能力上均有提升。然而,缺乏在可控且可复现条件下量化两代产品间实际推理性能差距的实证研究。本文提出 DEEP-GAP,一种基于 GDEV-AI 方法论的系统性评估框架,用于 GPU 推理分析。在相同配置和工作负载下,使用 PyTorch 与 TensorRT 对 ResNet18、ResNet50、ResNet101 在 FP32、FP16、INT8 精度模式下进行评估。结果表明,降低精度显著提升性能,其中 INT8 相较于 CPU 基线最高达 58 倍吞吐提升。L4 GPU 最高可达 T4 的 4.4 倍吞吐,且在批量大小 16 至 32 之间达到峰值效率,优化了对延迟敏感任务的吞吐-延迟权衡。而 T4 在大批次场景下仍具竞争力,适用于成本或能效优先的场景。DEEP-GAP 为现代推理部署提供了关于精度选择、批处理大小和硬件架构的实用指导。

原文摘要 · Abstract (English)

Modern datacenters increasingly rely on low-power, single-slot inference accelerators to balance performance, energy efficiency, and rack density constraints. The NVIDIA T4 GPU has become widely deployed due to strong performance per watt and mature software support. Its successor, the NVIDIA L4 GPU, introduces improvements in Tensor Core throughput, cache capacity, memory bandwidth, and parallel execution capability. However, limited empirical evidence quantifies the practical inference performance gap between these two generations under controlled and reproducible conditions. This work introduces DEEP-GAP, a systematic evaluation extending the GDEV-AI methodology to GPU inference. Using identical configurations and workloads, we evaluate ResNet18, ResNet50, and ResNet101 across FP32, FP16, and INT8 precision modes using PyTorch and TensorRT. Results show that reduced precision significantly improves performance, with INT8 achieving up to 58x throughput improvement over CPU baselines. L4 achieves up to 4.4x higher throughput than T4 while reaching peak efficiency at smaller batch sizes between 16 and 32, improving latency-throughput tradeoffs for latency-sensitive workloads. T4 remains competitive for large batch workloads where cost or power efficiency is important. DEEP-GAP provides practical guidance for selecting precision modes, batch sizes, and GPU architectures for modern inference deployments.

GPU性能推理优化模型部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。