根据输入动态跳过语音模型层，节省计算资源且性能不降。
Input Conditioned Layer Dropping in Speech Foundation Models
- 用输入特征和轻量选择网络决定跳过哪些层
- 在4个公开数据集上优于随机跳层，媲美甚至超过提前退出
- 适合资源波动的边缘设备部署，无需修改模型结构
为适应边缘和物联网设备中计算资源动态变化的需求，需构建可动态调整的语音基础模型。现有方法中的层跳过（$\ ext{LD}$）策略虽能降低推理开销，但存在层选择方式受限或需大幅修改网络结构的问题。本文提出输入驱动的$\ ext{LD}$方法，利用输入特征与轻量级层选择网络，自动确定最优的处理层组合。在两个预训练基础模型上，于4个语音与音频公开基准测试中进行大量实验，结果表明该方法显著优于随机跳层，性能与提前退出相当或更优。
原文摘要 · Abstract (English)
Curating foundation speech models for edge and IoT settings, where computational resources vary over time, requires dynamic architectures featuring adaptable reduction strategies. One emerging approach is layer dropping ($\mathcal{LD}$) which skips fraction of the layers of a backbone network during inference to reduce the computational load. This allows transforming static models into dynamic ones. However, existing approaches exhibit limitations either in the mode of selecting layers or by significantly modifying the neural architecture. To this end, we propose input-driven $\mathcal{LD}$ that employs the network's input features and a lightweight layer selecting network to determine the optimum combination of processing layers. Extensive experimentation on 4 speech and audio public benchmarks, using two different pre-trained foundation models, demonstrates the effectiveness of our approach, thoroughly outperforming random dropping and producing on-par (or better) results to early exit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。