arXiv:2607.19353cs.AIcs.LG2026-07

对比了在Intel TDX下NVIDIA H100的机密推理性能,发现大模型吞吐量下降但仍可用。

Benchmarking Confidential GPU Inference on NVIDIA H100 under Intel TDX

论文配图:Benchmarking Confidential GPU Inference on NVIDIA H100 under Intel TDX
图 1 · 摘自论文原文
  • 在Intel TDX机密实例中运行H100 GPU,测试机密与非机密模式下的推理性能。
  • 大模型Qwen3-30B-A3B机密模式下端到端延迟增27.8%,全局吞吐降21.1%。
  • 适合关注隐私保护部署的云服务设计者,尤其需规划大模型容量瓶颈。

机密计算正成为处理敏感输入或保护专有模型资产的AI推理工作负载的实际需求。然而,为GPU加速的大语言模型服务开启机密执行的性能开销仍依赖于具体工作负载,且具有重要的运营意义。本文在单个搭载于Intel TDX机密实例的NVIDIA H100 80GB GPU上,对标准非机密执行与机密计算模式进行了基准测试。评估使用Mistral-7B v0.1和Qwen3-30B-A3B两个代表性语言模型,测量首次令牌时间(TTFT)、端到端请求延迟、每请求生成吞吐率、全局吞吐率及在高并发下的闭环请求吞吐率。固定请求速率实验中,机密模式使Mistral-7B的平均TTFT增加21.8%,Qwen3-30B-A3B增加27.8%,全局吞吐率分别下降17.7%和21.1%。闭环并发实验中,吞吐差距维持在11.5%-20.2%之间,但大模型在机密模式下更早达到饱和拐点。结果表明,机密GPU推理可在负载下保持可用吞吐,但容量规划需考虑持续的吞吐损耗及大模型更早饱和的现象。

原文摘要 · Abstract (English)

Confidential computing is becoming a practical deployment requirement for AI inference workloads that process sensitive inputs or protect proprietary model assets. However, the performance cost of enabling confidential execution for GPU-accelerated large language model serving remains workload dependent and operationally important. This paper presents a benchmark study comparing standard non-confidential execution with confidential computing mode on a single NVIDIA H100 80GB GPU hosted in an Intel TDX confidential instance. The evaluation uses two representative language models, Mistral-7B v0.1 and Qwen3-30B-A3B, and measures time to first token, end-to-end request latency, per-request token generation throughput, global token throughput, and closed-loop request throughput under increasing concurrency. In fixed request-rate experiments, confidential mode increases average TTFT by 21.8% for Mistral-7B and 27.8% for Qwen3-30B-A3B, while global token throughput drops by 17.7% and 21.1%, respectively. In closed-loop concurrency experiments, throughput gaps remain in the 11.5-20.2% range, but the larger model reaches its saturation knee earlier under confidential mode. The results suggest that confidential GPU inference can retain usable throughput under load, but capacity planning must account for both the steady throughput penalty and the earlier saturation behavior observed for larger models.

机密计算大模型推理性能评估H100

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。