轻量级语音增强模型IMSE通过新注意力和卷积结构，在参数减少16.8%下保持顶尖性能。
IMSE: Efficient U-Net-based Speech Enhancement using Inception Depthwise Convolution and Amplitude-Aware Linear Attention
- 用幅度感知线性注意力替代复杂模块，高效建模全局特征。
- 引入分层深度卷积分解大核操作，大幅降低参数冗余。
- 适合资源受限设备部署，兼顾模型大小与语音质量。
在资源受限设备上实现轻量化设计与高性能的平衡仍是语音增强任务的核心挑战。现有先进方法MUSE以仅0.513M参数建立强基线，采用多路径增强泰勒变换（MET）注意力与可变形嵌入（DE）。但深入分析发现，MET依赖复杂的“近似-补偿”机制以弥补泰勒展开注意力的缺陷，而可变形嵌入的偏移计算带来额外计算负担。本文提出IMSE，一种系统优化的超轻量网络：1）以幅度感知线性注意力（MALA）替代MET，显式保留查询向量幅值信息，消除“忽略幅度”的问题，无需辅助补偿分支即可实现高效全局建模；2）以级联深度卷积（IDConv）替代DE，借鉴Inception思想，将大核操作分解为并行的方形、水平与垂直条带分支，显著降低参数冗余。在VoiceBank+DEMAND数据集上的大量实验表明，相比MUSE基线，IMSE参数量减少16.8%（从0.513M降至0.427M），同时在PESQ指标上达到3.373，性能与当前最优水平相当。本研究为超轻量语音增强中模型规模与语音质量的权衡树立了新标杆。
原文摘要 · Abstract (English)
Achieving a balance between lightweight design and high performance remains a significant challenge for speech enhancement (SE) tasks on resource-constrained devices. Existing state-of-the-art methods, such as MUSE, have established a strong baseline with only 0.51M parameters by introducing a Multi-path Enhanced Taylor (MET) transformer and Deformable Embedding (DE). However, an in-depth analysis reveals that MUSE still suffers from efficiency bottlenecks: the MET module relies on a complex "approximate-compensate" mechanism to mitigate the limitations of Taylor-expansion-based attention, while the offset calculation for deformable embedding introduces additional computational burden. This paper proposes IMSE, a systematically optimized and ultra-lightweight network. We introduce two core innovations: 1) Replacing the MET module with Amplitude-Aware Linear Attention (MALA). MALA fundamentally rectifies the "amplitude-ignoring" problem in linear attention by explicitly preserving the norm information of query vectors in the attention calculation, achieving efficient global modeling without an auxiliary compensation branch. 2) Replacing the DE module with Inception Depthwise Convolution (IDConv). IDConv borrows the Inception concept, decomposing large-kernel operations into efficient parallel branches (square, horizontal, and vertical strips), thereby capturing spectrogram features with extremely low parameter redundancy. Extensive experiments on the VoiceBank+DEMAND dataset demonstrate that, compared to the MUSE baseline, IMSE significantly reduces the parameter count by 16.8\% (from 0.513M to 0.427M) while achieving competitive performance comparable to the state-of-the-art on the PESQ metric (3.373). This study sets a new benchmark for the trade-off between model size and speech quality in ultra-lightweight speech enhancement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。