arXiv:2606.06510cs.ARcs.AI2026-06被引 1

用FP8矩阵运算取代传统双精度计算,突破高性能计算瓶颈

FP8 is All You Need (Part 1): Debunking Hardware FP64 as the HPC Holy Grail (June 13th version)

  • 以FP8张量核心为基础构建所有科学计算内核
  • 实测显示在B300 GPU上FP8性能达多PFLOPS,FP64仅1.3 TFLOPS
  • 适合关注硬件算力重构与高效算法设计的研究者

传统高性能计算认为原生硬件双精度(FP64)是科学计算不可替代的基础。然而,在NVIDIA B300及后续的AI优化GPU上,原生FP64吞吐量已降至约1.3 TFLOPS,而FP8张量吞吐量却达到多PFLOPS。本文主张:更进一步,双精度科学计算只需建立在FP8张量核心矩阵乘法这一单一计算原语之上即可。所有典型内核——稠密与稀疏线性代数、谱变换、有限差分模板等——均可通过基于中国剩余定理的Ozaki方案II,转化为一系列FP8矩阵操作;唯一非FP8的运算仅为重建时的固定宽度整数累加。因此,原生FP64从硬件要求降级为通过组合FP8原语获得的精度保障。我们构建五层架构体系(FP8操作、Ozaki II、基础内核即伯克利‘侏儒’、复合求解器、完整应用),并通过展示每个‘侏儒’的归约过程来验证该主张。提出可检验的测试工具——张量-内存平衡(TME)模型,扩展屋顶图并引入模拟参数(alpha, beta, gamma)。识别出寄存器融合是使模拟保持内存受限的关键机制,并在B300与Rubin平台上投影恢复的FP64性能,相较H100基准进行评估。通过配套的FFT分析和补偿归约完成内核覆盖。该模型本可能给出否定结论,但最终在各类内核及其组合中均通过验证。这是两部分计划中的理论部分,后续将通过真实硅片实现验证。

原文摘要 · Abstract (English)

Conventional HPC holds that native hardware FP64 is the irreducible foundation of scientific computing. On AI-optimized GPUs of the NVIDIA B300 generation and beyond, native FP64 throughput has collapsed to ~1.3 TFLOPS even as FP8 tensor throughput has grown to multiple PFLOPS. We argue something stronger than that this is survivable: the FP8 tensor-core matrix-multiply is the sole computational primitive on which double-precision scientific computing needs to be built. Every canonical kernel -- dense and sparse linear algebra, spectral transforms, stencils -- and every application composing them reduces, via the Chinese Remainder Theorem-based Ozaki Scheme II, to sequences of FP8 matrix operations; the only non-FP8 arithmetic is a bounded, fixed-width integer accumulation at reconstruction. Native FP64 is thereby demoted from a hardware requirement to a derived accuracy guarantee obtained by composition over the FP8 primitive. We organize the claim as a five-layer hierarchy -- the FP8 op, Ozaki II, the basic kernels or Berkeley "dwarfs", composite solvers, and full applications -- and, because the dwarf taxonomy already spans scientific computing, establish it by exhibiting the reduction for every dwarf rather than a sample. The claim is falsifiable, and we build the instrument that tests it: a Tensor-Memory Equilibrium (TME) model extending the Roofline with emulation parameters (alpha, beta, gamma). We identify register-level fusion as the mechanism that keeps emulation memory-bound, project recovered FP64 performance across B300 and Rubin against an H100 baseline, and close the kernel coverage with a companion FFT analysis and compensated reductions. The model could have returned a negative verdict; instead it passes across the dwarfs and their compositions. This is the analytical half of a two-part program, with a follow-on implementation to validate the thesis on real silicon.

FP8高性能计算算法优化张量核心

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。