用语音增强模型的动态剪枝掩码，同时估算语音属性，省去额外模型。
From Diet to Free Lunch: Estimating Auxiliary Signal Properties using Dynamic Pruning Masks in Speech Enhancement Networks
- 利用动态通道剪枝掩码提取语音信号特征，无需额外模块。
- 在VAD任务上准确率达93%，噪声分类84%，基频估计相关系数0.86。
- 仅需加权求和即可预测，计算开销几乎为零，适合移动端部署。
音频设备中的语音增强（SE）常依赖辅助模块（如语音活动检测VAD、信噪比估计、声学场景分类）以实现鲁棒的上下文感知行为和流畅体验。尽管这些任务也使用深度学习，但将额外模型部署在设备端计算成本过高，而云端推理则增加延迟并影响隐私。先前研究采用动态通道剪枝（DynCP）通过输入自适应禁用特定通道来降低计算量。本文探究能否从这些内部剪枝掩码中估计有用信号属性，从而消除对独立模型的需求。结果表明，简单的可解释预测器在VAD上达到93%准确率，噪声分类84%，基频（F0）估计的R2达0.86。使用二值掩码时，预测仅需加权求和，带来可忽略的额外开销。本工作贡献双重：一是通过下游预测任务揭示DynCP模型的涌现行为与学习内容；二是将DynCP重新定位为高效语音增强与信号属性同步估计的综合性解决方案。
原文摘要 · Abstract (English)
Speech Enhancement (SE) in audio devices is often supported by auxiliary modules for Voice Activity Detection (VAD), SNR estimation, or Acoustic Scene Classification to ensure robust context-aware behavior and seamless user experience. Just like SE, these tasks often employ deep learning; however, deploying additional models on-device is computationally impractical, whereas cloud-based inference would introduce additional latency and compromise privacy. Prior work on SE employed Dynamic Channel Pruning (DynCP) to reduce computation by adaptively disabling specific channels based on the current input. In this work, we investigate whether useful signal properties can be estimated from these internal pruning masks, thus removing the need for separate models. We show that simple, interpretable predictors achieve up to 93% accuracy on VAD, 84% on noise classification, and an R2 of 0.86 on F0 estimation. With binary masks, predictions reduce to weighted sums, inducing negligible overhead. Our contribution is twofold: on one hand, we examine the emergent behavior of DynCP models through the lens of downstream prediction tasks, to reveal what they are learning; on the other, we repurpose and re-propose DynCP as a holistic solution for efficient SE and simultaneous estimation of signal properties.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。