WebGPU推理时每操作开销高达95微秒，远超预期，影响性能优化方向。
Characterizing WebGPU Dispatch Overhead for LLM Inference Across Four GPU Vendors, Three Backends, and Three Browsers
- 设计串行调度法，发现传统测试高估开销近20倍
- 实际单次调用开销24-71微秒，含Python成本达95微秒
- 不同后端差异大，适合关注浏览器端LLM部署者
WebGPU的安全设计导致每次操作需验证，神经网络推理中大量小调度使开销累积。我们系统评估了四种显卡厂商（NVIDIA、AMD、Apple、Intel）、两种原生实现（Dawn、wgpu-native）、三种浏览器（Chrome、Safari、Firefox）及两个模型尺寸（Qwen2.5-0.5B和1.5B）在批量大小为1下的推理开销。提出串行调度方法，揭示传统单操作测试将开销高估约20倍。仅WebGPU API开销为24-36μs（Vulkan）和32-71μs（Metal），总开销（含Python）约为95μs。Vulkan上核融合提升吞吐53%，而CUDA无提升，证明操作开销是关键差异点。测试覆盖三大操作系统（Linux、Windows、macOS）。构建torch-webgpu，基于PrivateUse1的PyTorch后端与FX转WebGPU编译器，在参考平台达到CUDA性能的11%-12%。同精度下，RTX PRO 2000比WebGPU快1.4倍，尽管算力仅为RTX 5090的1/6。后端选择是主导因素，同一后端内实现差异也显著（如Metal差2.2倍）。结论：当前批大小为1的调度密集型流程中，操作开销始终主导，无论内核质量如何。所有代码、基准与原始数据开源。
原文摘要 · Abstract (English)
WebGPU's security-focused design imposes per-operation validation that compounds across the many small dispatches in neural network inference, yet the true cost of this overhead is poorly characterized. We present a systematic characterization of WebGPU dispatch overhead for LLM inference at batch size 1, spanning four GPU vendors (NVIDIA, AMD, Apple, Intel), two native implementations (Dawn, wgpu-native) and three browsers (Chrome, Safari, Firefox), and two model sizes (Qwen2.5-0.5B and 1.5B). Our primary contribution is a sequential-dispatch methodology that reveals naive single-operation benchmarks overestimate dispatch cost by ${\sim}20\times$. The true per-dispatch cost of WebGPU API overhead alone is 24-36 $μ$s on Vulkan and 32-71 $μ$s on Metal, while the total per-operation overhead including Python cost is ${\sim}95$~$μ$s, which turns out to be a distinction critical for optimization. On Vulkan, kernel fusion improves throughput by 53%, while CUDA fusion provides no benefit, confirming that per-operation overhead is a primary differentiator. LLM inference was tested across three major operating systems (Linux, Windows, macOS). We built $\texttt{torch-webgpu}$, a PrivateUse1-based out-of-tree PyTorch backend and an FX-to-WebGPU compiler, which on our reference platform achieves 11--12% of CUDA performance. At dtype-matched float32, RTX PRO 2000 achieves 1.4$\times$ WebGPU's throughput despite ${\sim}6\times$ less compute than RTX 5090. For dispatch overhead, backend choice is the dominant factor, although implementation choice also matters substantially within a backend (2.2$\times$ for Metal). In terms of dispatch vs kernel compute efficiency, we conclude that at batch=1 with the current dispatch-heavy pipeline, per-operation overhead dominates regardless of kernel quality. All code, benchmarks, and raw data are open source.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。