用轻量MLP替代Transformer，实现高效语音验证模型压缩
SV-Mixer: Replacing the Transformer Encoder with Lightweight MLPs for Self-Supervised Model Compression in Speaker Verification
- 用三个轻量模块替代Transformer，处理时序、上下文和频谱特征
- 参数和计算量减半，75%压缩下仍接近教师模型性能
- 适合移动端实时语音验证，硬件友好且准确率高
自监督学习使语音验证准确率接近顶尖水平，但主流SSL编码器中的Transformer骨干网络阻碍了设备端和实时部署。已有压缩方法仅减少层数或宽度，仍保留自注意力的二次计算开销。本文提出SV-Mixer，首个完全基于MLP的学生编码器用于SSL蒸馏。它用三个轻量模块替代Transformer：多尺度混合（Multi-Scale Mixing）捕捉多分辨率时序特征，局部-全局混合（Local-Global Mixing）建模帧到话语的上下文，组通道混合（Group Channel Mixing）处理频谱子空间。从WavLM蒸馏得到的SV-Mixer比Transformer学生模型提升14.6%，参数与GMACs均减少超一半；在75%压缩率下，性能接近教师模型。结果表明，无注意力的SSL学生模型可实现教师级精度，且具备硬件友好特性，为鲁棒的设备端语音验证打开新路径。
原文摘要 · Abstract (English)
Self-supervised learning (SSL) has pushed speaker verification accuracy close to state-of-the-art levels, but the Transformer backbones used in most SSL encoders hinder on-device and real-time deployment. Prior compression work trims layer depth or width yet still inherits the quadratic cost of self-attention. We propose SV-Mixer, the first fully MLP-based student encoder for SSL distillation. SV-Mixer replaces Transformer with three lightweight modules: Multi-Scale Mixing for multi-resolution temporal features, Local-Global Mixing for frame-to-utterance context, and Group Channel Mixing for spectral subspaces. Distilled from WavLM, SV-Mixer outperforms a Transformer student by 14.6% while cutting parameters and GMACs by over half, and at 75% compression, it closely matches the teacher's performance. Our results show that attention-free SSL students can deliver teacher-level accuracy with hardware-friendly footprints, opening the door to robust on-device speaker verification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。