用低秩近似和FP8加速，让大规模矩阵乘法更快更省内存。
Low-Rank GEMM: Efficient Matrix Multiplication via Low-Rank Approximation with FP8 Acceleration
- 通过低秩分解降低计算复杂度，结合FP8精度与智能内核选择。
- 在4090上实现378 TFLOPS，比PyTorch FP32快7.8倍，节省75%内存。
- 自动适配硬件，适合大规模矩阵运算场景，尤其适用于大模型推理。
大规模矩阵乘法是现代机器学习的核心任务，但传统方法存在立方级计算复杂度（如n×n矩阵为𝒪(n³)）。本文提出Low-Rank GEMM，利用低秩矩阵近似实现亚二次复杂度，同时通过FP8精度和智能内核选择保持硬件加速性能。在NVIDIA RTX 4090上，该实现对最大尺寸为N=20480的矩阵达到最高378 TFLOPS，相比PyTorch FP32实现75%内存节省和7.8倍加速。系统根据矩阵特征与可用加速器自动选择最优分解方法（SVD、随机SVD）与精度级别。全面基准测试表明，当矩阵尺寸N≥10240时，Low-Rank GEMM成为最快方案，其优势源于内存带宽优化而非计算捷径，超越传统cuBLAS实现。
原文摘要 · Abstract (English)
Large matrix multiplication is a cornerstone of modern machine learning workloads, yet traditional approaches suffer from cubic computational complexity (e.g., $\mathcal{O}(n^3)$ for a matrix of size $n\times n$). We present Low-Rank GEMM, a novel approach that leverages low-rank matrix approximations to achieve sub-quadratic complexity while maintaining hardware-accelerated performance through FP8 precision and intelligent kernel selection. On a NVIDIA RTX 4090, our implementation achieves up to 378 TFLOPS on matrices up to $N=20480$, providing 75\% memory savings and $7.8\times$ speedup over PyTorch FP32 for large matrices. The system automatically adapts to hardware capabilities, selecting optimal decomposition methods (SVD, randomized SVD) and precision levels based on matrix characteristics and available accelerators. Comprehensive benchmarking on NVIDIA RTX 4090 demonstrates that Low-Rank GEMM becomes the fastest approach for matrices $N\geq10240$, surpassing traditional cuBLAS implementations through memory bandwidth optimization rather than computational shortcuts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。