用FP8张量核实现全精度浮点3D FFT，突破显卡算力瓶颈。
FP8 is All You Need (Part 2): Efficient Ozaki-Bailey Style FFT Through Tensor-core Garner Reformulation and Kulisch Escape Route
- 将3D FFT拆解为多步1D FFT，借助FP8张量核加速计算
- 在B300上实现1024³ 3D FFT仅需约18毫秒，接近内存上限
- 提出库利什整数算术方案，保障全精度同时避开算力限制
NVIDIA Blackwell Ultra (B300) 的双精度浮点向量吞吐量降至约1.3 TFLOPS/GPU，约为B200的1/30，远低于内存带宽受限场景下的算力门槛。通过奥扎基方案II框架，利用分段尾数的中国剩余定理重构，将密集矩阵乘法路由至FP8张量核心，恢复双精度等效性能。此前部分（1）涵盖密集GEMM、批量GEMV、模板和稀疏矩阵向量乘；本文补充第五个基础算子：三维FFT。我们提出奥扎基-贝利FFT，基于贝利六步分解法，将所有一维FFT GEMM运算部署于FP8张量核心。当内层因子k≈√N（N=1024时k=32）时，满足k≪r²条件，第三项TME参数γ（重构延迟）成为瓶颈而非可摊销。加纳重构分为两阶段：阶段A（FP8/INT8张量核心完成内积，1024³耗时约1毫秒），阶段B（逐输出归约）。我们识别出库利什定点完全算术可作为阶段B的重构形式，使计算全程运行于INT32 SIMT流水线，同时保持完整双精度精度。推导出带宽对齐的理论下限：原生双精度下限为1.56×B_HBM（8 TB/s带宽下12.5 TF），B300的1.3 TF低约10倍；鲁宾预测值33 TF误差在4%内。库利什路径需满足两个下限：INT32子下限8.25×B_HBM与FP8下限170×B_HBM，B300均达标。投影显示1024³ 3D FFT满双精度耗时约18毫秒，基本贴合12.9毫秒内存天花板。若实际验证成立，仅靠软件即可使B300支持全双精度FFT，推动libKulisch库与基准测试计划。
原文摘要 · Abstract (English)
NVIDIA's Blackwell Ultra (B300) cuts FP64 vector throughput to ~1.3 TFLOPS per GPU, roughly 30x below B200 and well below the level at which bandwidth-limited FP64 workloads stay memory-bound. The Ozaki Scheme II framework recovers FP64-equivalent throughput by routing dense matrix multiply through FP8 tensor cores with a mantissa-sliced Chinese-remainder reconstruction. A companion Part (1) paper covers dense GEMM, batched GEMV, stencils, and SpMV; this paper adds the fifth canonical primitive, the 3-D FFT. We present Ozaki-Bailey FFT, an emulated 3-D FFT via the Bailey six-step decomposition with both 1-D FFT GEMMs on FP8 tensor cores. Bailey's small inner factor k ~ sqrt(N) (k=32 for N=1024) puts the kernel in the regime k << r^2, where the third TME parameter gamma (reconstruction latency) binds rather than amortising. Garner reconstruction splits into Phase A (inner products on FP8/INT8 tensor cores, ~1 ms for 1024^3 on B300) and Phase B (per-output reduction). We identify Kulisch fixed-point complete arithmetic as a Phase B reformulation that keeps full FP64 accuracy while running entirely on the INT32 SIMT pipe. We derive closed-form bandwidth-parity floors. The native FP64 floor is 1.56*B_HBM (12.5 TF at 8 TB/s): B300's 1.3 TF sits ~10x below, Rubin's 33 TF within 4%. The Kulisch escape route needs an INT32 sub-floor 8.25*B_HBM and an FP8 floor 170*B_HBM; B300 meets both. The projection is ~18 ms for 1024^3 at full FP64, essentially the 12.9 ms memory roof. A GPU meets memory-roof FFT parity if it satisfies either the native floor or both Kulisch floors. If the projection holds in practice, B300 becomes viable for full-FP64 FFT through software alone, motivating a libKulisch library and benchmark campaign.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。