对比英伟达与苹果芯片在大模型本地推理中的性能与效率，揭示硬件架构的关键差异。
Silicon Showdown: Performance, Efficiency, and Ecosystem Barriers in Consumer-Grade LLM Inference
- 分析英伟达与苹果硅芯片在部署700亿以上参数模型时的架构权衡。
- 英伟达用新量化格式可提速1.6倍，但需牺牲启动延迟；苹果统一内存实现线性扩展。
- 苹果芯片能效比高23倍，适合长期本地运行，但生态闭锁限制灵活性。
本地大语言模型（LLM）推理的运行环境已从轻量模型转向参数量超700亿的数据中心级模型，对消费级硬件带来严峻系统挑战。本文系统性地分析了英伟达与苹果硅生态，具体刻画了在部署这些大型模型时所需的独特架构权衡。在英伟达黑瓦尔架构上，我们发现TensorRT-LLM栈中存在关键的“后端二分法”：尽管新的NVFP4量化格式相比优化后的BF16基线提升1.6倍吞吐量（151 tokens/s vs. 92 tokens/s），但实现此性能需面对复杂的运行时约束，以牺牲启动延迟换取生成速度。此外，我们定义了700亿+模型的“显存墙”问题：在独立显卡上，用户面临两难选择——采用激进量化（如Q2）会损害模型智能以适应显存，或依赖PCIe瓶颈的CPU卸载，使吞吐量较全GPU执行下降超过90%。相反，苹果统一内存架构（UMA）绕过这些瓶颈，支持800亿参数模型在实用4比特精度下实现线性扩展。这一架构差异延伸至运行可持续性，苹果芯片的SoC设计展现出高达23倍的能效优势（每焦耳令牌数）。结论表明，消费级推理的最优硬件由计算密度（英伟达）与内存容量（苹果）的复杂互动决定，受制于专有量化工作流带来的显著生态系统摩擦。
原文摘要 · Abstract (English)
The operational landscape of local Large Language Model (LLM) inference has shifted from lightweight models to datacenter-class weights exceeding 70B parameters, creating profound systems challenges for consumer hardware. This paper presents a systematic empirical analysis of the Nvidia and Apple Silicon ecosystems, specifically characterizing the distinct intra-architecture trade-offs required to deploy these massive models. On the Nvidia Blackwell architecture, we identify a critical "Backend Dichotomy" within the TensorRT-LLM stack: while the new NVFP4 quantization format delivers a 1.6x throughput advantage over optimized BF16 baselines (151 tokens/s vs. 92 tokens/s), realizing this performance requires navigating complex runtime constraints that trade startup latency for generation speed. Furthermore, we characterize the "VRAM Wall" for 70B+ models: on discrete GPUs, users face a destructive choice between aggressive quantization (e.g., Q2) that degrades model intelligence to fit in VRAM, or PCIe-bottlenecked CPU offloading, which reduces throughput by over 90% compared to full-GPU execution. Conversely, Apple's Unified Memory Architecture (UMA) circumvents these bottlenecks, enabling linear scaling for 80B parameter models at practical 4-bit precisions. This architectural divergence extends to operational sustainability, where Apple's SoC design demonstrates up to a 23x advantage in energy efficiency (tokens/joule). We conclude that for consumer-grade inference, the optimal hardware is defined by a complex interplay between compute density (Nvidia) and memory capacity (Apple), moderated by the significant "ecosystem friction" of proprietary quantization workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。