用GEMM替代FFT实现更高效的音频前端，显著降低延迟与能耗。
MelT: A Portable, Single-GEMM Mel Audio Frontend via Non-Uniform DFT with Measured Latency and Energy Gains on GPUs
- 将梅尔滤波器组转换为非均匀DFT预计算基，通过GEMM一次性完成特征提取
- 在苹果A18 Pro到英伟达H100上实现1.64至3.29倍延迟降低，能耗减少最多3.03倍
- 适合部署在边缘与数据中心的神经音频模型，尤其关注能效与端侧推理
现代神经音频模型依赖加速器运行，其峰值吞吐量来自密集矩阵乘法。然而传统声学前端（短时傅里叶变换+稀疏梅尔聚合）仍以快速傅里叶变换（FFT）为中心，存在执行开销，难以匹配推理栈中密集线性代数的性能。本文提出MelT，一种可移植的单阶段梅尔前端，通过预计算梅尔间距的非均匀离散傅里叶变换（NDFT）基，并利用通用矩阵乘法（GEMM）对时域帧进行投影。该设计核心是将梅尔特征提取解耦于厂商特定的FFT原语，下沉至加速器已优化的矩阵乘法基础架构。尽管运算量高于传统FFT流水线，但在神经音频前端的紧凑分辨率下，从苹果A18 Pro到英伟达H100均实现1.64至3.29倍延迟降低，主动能耗减少最多3.03倍。所有增益均为同平台对比，经任务级验证：冻结的中等及以上规模Whisper模型词错误率无统计差异；VoxCeleb1说话人属性分类表现相当。扩展版MFCCT在临床呼吸功能不全分类任务（SPIRA）中保持有效性，优于原始MFCC基线。结果表明，在所评估加速器的实际场景中，硬件对齐比算术计数更能决定特征提取的实际成本。
原文摘要 · Abstract (English)
Modern neural audio models run on accelerators whose peak throughput comes from dense matrix multiplication, increasingly at the edge and in datacenters. The conventional acoustic frontend, however -- a Short-Time Fourier Transform (STFT) followed by sparse Mel aggregation -- remains a multi-stage pipeline centered on the Fast Fourier Transform (FFT), with execution overheads unlike the dense linear algebra dominating the inference stack. This work introduces MelT, a portable single-stage Mel frontend that precomputes Mel-spaced Non-Uniform Discrete Fourier Transform (NDFT) bases and applies them to time-domain frames through General Matrix Multiplication (GEMM). The contribution is a computational design principle: decoupling Mel feature extraction from vendor-specific FFT primitives and lowering it onto the matrix-multiplication substrate accelerators already optimize. It is not a new spectral operator. MelT's direct projection performs more arithmetic than the FFT pipeline. Yet in the compact-resolution regime of neural audio frontends, it achieves a 1.64-times to 3.29-times latency reduction and up to a 3.03-times reduction in measured active energy, from the Apple A18 Pro to the NVIDIA H100. All gains are within-platform comparisons, accompanied by task-level validation. Word error rate stays statistically equivalent to the native frontend's on frozen Whisper models of medium size and larger; speaker-attribute classification on VoxCeleb1 is non-inferior. The cepstral extension MFCCT preserves utility on a clinical respiratory-insufficiency classification task (SPIRA) while improving on the MFCC baseline. These results indicate that, in practical regimes on the accelerators evaluated here, hardware alignment rather than arithmetic count can govern the realized cost of feature extraction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。