无需预训练权重，支持任意采样率与长度的音频分类新框架
AMAuT: A Flexible and Efficient Multiview Audio Transformer Framework Trained from Scratch
- 通过多视角增强学习实现鲁棒音频表征
- 在5个基准上最高达99.8%准确率，仅需不足3%的显卡时长
- 适合资源受限场景下的高性能音频分类应用
近期基础模型如SSAST、EAT、HuBERT、Qwen-Audio和Audio Flamingo在标准音频基准上表现优异，但受限于固定输入速率和长度，复用性差。本文提出从零训练的增强驱动多视角音频变换器（AMAuT），摆脱对预训练权重的依赖，支持任意采样率与音频长度。AMAuT包含四个核心组件：(1) 增强驱动的多视角学习以提升鲁棒性；(2) 一维卷积瓶颈（conv1 + conv7 + conv1）实现稳定的时间编码；(3) 双向上下文表示的双CLS + TAL标记；(4) 测试时适配/增强（TTA^2）提升推理可靠性。在AudioMNIST、SpeechCommands V1 & V2、VocalSound和CochlScene五个公开基准上的实验表明，AMAuT最高达到99.8%准确率，所耗GPU时间不足同类预训练模型的3%。因此，AMAuT为计算资源受限环境提供了高效灵活的先进音频分类方案。
原文摘要 · Abstract (English)
Recent foundational models, SSAST, EAT, HuBERT, Qwen-Audio, and Audio Flamingo, achieve top-tier results across standard audio benchmarks but are limited by fixed input rates and durations, hindering their reusability. This paper introduces the Augmentation-driven Multiview Audio Transformer (AMAuT), a training-from-scratch framework that eliminates the dependency on pre-trained weights while supporting arbitrary sample rates and audio lengths. AMAuT integrates four key components: (1) augmentation-driven multiview learning for robustness, (2) a conv1 + conv7 + conv1 one-dimensional CNN bottleneck for stable temporal encoding, (3) dual CLS + TAL tokens for bidirectional context representation, and (4) test-time adaptation/augmentation (TTA^2) to improve inference reliability. Experiments on five public benchmarks, AudioMNIST, SpeechCommands V1 & V2, VocalSound, and CochlScene, show that AMAuT achieves accuracies up to 99.8% while consuming less than 3% of the GPU hours required by comparable pre-trained models. Thus, AMAuT presents a highly efficient and flexible alternative to large pre-trained models, making state-of-the-art audio classification accessible in computationally constrained settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。