提出轻量级音频模型HEAR，高效提取音频特征并保持高精度。
A Human-Inspired Decoupled Architecture for Efficient Audio Representation Learning
- 分模块设计：声学模型专注局部特征，任务模型整合全局语义。
- 仅需1500万参数和9.47亿次浮点运算，计算成本仅为传统模型的零头。
- 适合边缘设备部署，适用于语音识别、音频分类等实际场景。
自监督学习虽已革新音频表征，但标准Transformer参数过多且计算复杂度为二次方，难以在资源受限设备上部署。为此，我们提出受人类认知启发的高效音频表征模型HEAR，将处理流程解耦为两个专用模块：用于局部特征提取的声学模型与用于全局语义整合的任务模型。结合通过知识蒸馏训练的声学分词器，实现稳健的掩码音频建模（MAM）。大量实验表明，HEAR推理时仅需1500万参数和9.47亿次浮点运算，远低于传统基础模型（通常需8500万至9400万参数）。尽管效率极高，其在多种音频分类基准上仍表现优异。代码与预训练模型已开源于https://github.com/HarunoriKawano/HEAR。
原文摘要 · Abstract (English)
While self-supervised learning (SSL) has revolutionized audio representation, the excessive parameterization and quadratic computational cost of standard Transformers limit their deployment on resource-constrained devices. To address this bottleneck, we propose HEAR (Human-inspired Efficient Audio Representation), a novel decoupled architecture. Inspired by the human cognitive ability to isolate local acoustic features from global context, HEAR splits the processing pipeline into two dedicated modules: an Acoustic Model for local feature extraction and a Task Model for global semantic integration. Coupled with an Acoustic Tokenizer trained via knowledge distillation, our approach enables robust Masked Audio Modeling (MAM). Extensive experiments demonstrate that HEAR requires only 15M parameters and 9.47 GFLOPs for inference, operating at a fraction of the computational cost of conventional foundation models (which typically require 85M-94M parameters). Despite this high efficiency, HEAR achieves highly competitive performance across diverse audio classification benchmarks. The code and pre-trained models are available at https://github.com/HarunoriKawano/HEAR
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。