FAST融合卷积与注意力机制，用极少量参数实现音频分类实时高效
FAST: Fast Audio Spectrogram Transformer
- 结合CNN局部特征提取与Transformer全局建模优势
- 在ADIMA和AudioSet上达顶尖性能，参数量少至150倍
- 适合移动端或实时语音检测场景，训练更稳定
在音频分类任务中，构建高效且鲁棒的模型对实时应用至关重要。受MobileViT设计启发，本文提出FAST（Fast Audio Spectrogram Transformer），一种融合卷积神经网络（CNN）与Transformer的新架构，兼顾CNN的局部特征提取效率与Transformer的全局上下文建模能力，实现强大而轻量的模型，适用于实时或移动端场景。此外，引入Lipschitz连续注意力机制以提升训练稳定性并加速收敛。我们在ADIMA数据集（多语言实时辱骂与骚扰检测语料库）和传统AudioSet上评估了FAST，结果表明其在两项分类任务中均达到当前最优性能，在某些情况下超越现有基准，同时参数量减少多达150倍。
原文摘要 · Abstract (English)
In audio classification, developing efficient and robust models is critical for real-time applications. Inspired by the design principles of MobileViT, we present FAST (Fast Audio Spectrogram Transformer), a new architecture that combines convolutional neural networks (CNNs) and transformers to capitalize on the strengths of both. FAST integrates the local feature extraction efficiencies of CNNs with the global context modeling capabilities of transformers, resulting in a model that is powerful yet lightweight, well-suited to a real-time or mobile use case. Additionally, we incorporate Lipschitz continuous attention mechanisms to improve training stability and accelerate convergence. We evaluate FAST on the ADIMA dataset, a multilingual corpus towards real-time profanity and abuse detection, as well as on the more traditional AudioSet. Our results show that FAST achieves state-of-the-art performance on both the ADIMA and AudioSet classification tasks and in some cases surpasses existing benchmarks while using up to 150x fewer parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。