优化哈达玛变换算法，利用GPU张量核心实现更快推理。
HadaCore: Tensor Core Accelerated Hadamard Transform Kernel
- 基于递归结构改进快速哈达玛变换，适配GPU张量核心
- 在A100/H100上最高提速3.6倍，实测加速1.1-1.4倍
- 支持FP16/BF16，量化后仍保持模型精度
我们提出HadaCore，一种针对现代GPU中张量核心优化的快速沃尔什-哈达玛变换（FWHT）算法。该算法沿用原始FWHT的递归结构，保持相同的渐近时间复杂度，同时通过硬件感知的工作分解方式，充分利用张量核心加速，缓解计算与数据交换瓶颈。在Nvidia A100和H100 GPU上，相比现有最优实现，速度提升1.1–1.4倍和1.0–1.3倍，峰值分别达到3.5倍和3.6倍。此外，在使用FP16或BF16时，该实现具备数值准确性，用于量化（FP8）注意力的端到端Llama3推理时，可在MMLU基准测试中实现相近的准确率。
原文摘要 · Abstract (English)
We present HadaCore, a modified Fast Walsh-Hadamard Transform (FWHT) algorithm optimized for the Tensor Cores present in modern GPU hardware. HadaCore follows the recursive structure of the original FWHT algorithm, achieving the same asymptotic runtime complexity but leveraging a hardware-aware work decomposition that benefits from Tensor Core acceleration. This reduces bottlenecks from compute and data exchange. On Nvidia A100 and H100 GPUs, HadaCore achieves speedups of 1.1-1.4x and 1.0-1.3x, with a peak gain of 3.5x and 3.6x respectively, when compared to the existing state-of-the-art implementation of the original algorithm. We also show that when using FP16 or BF16, our implementation is numerically accurate, enabling comparable accuracy on MMLU benchmarks when used in an end-to-end Llama3 inference run with quantized (FP8) attention.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。