通过注意力机制解析语音防伪中多特征与模型层的关联，提升可解释性。
MOSAIC: Interpretable Multi-Token Cross-Attention of Biophonetic and Self-Supervised Representations for Unified Voice Anti-Spoofing
- 将152维生物音素特征拆分为6组语义查询令牌，与WavLM模型层进行交叉注意力
- 在ASVspoof 2019 LA/PA上达到EER 1.93%/1.98%，接近最优专用模型表现
- 可视化特征-层对齐关系，揭示真声与伪造声激活模式差异，适合安全可信场景
语音防伪主流方法融合自监督（SSL）骨干网络（如WavLM）与手工特征，但此类融合通常缺乏线索到模型层的透明性，简单拼接也限制了跨模态学习。本文提出MOSAIC（基于整合交叉注意力的多令牌语音防伪），一个可解释的多令牌交叉注意力框架：将152维生物音素特征向量拆分为六组语义分组的查询令牌（Praat、相位、LFCC均值/方差、子带均值/方差），并在十三个均值-方差池化的WavLM-Large Transformer层上作为键值进行注意力计算。生成的6×13注意力矩阵可可视化线索-层对齐；对每令牌激活的z-score分析表明，生物音素/相位令牌更激活于真实语音，而谱图/通道令牌更激活于伪造语音，实现线索与层级别的归因，超越以往融合方法。联合训练焦点损失、双域对抗分类器（LA/PA）及仅真声的变分自编码器正则化器后，MOSAIC在ASVspoof 2019 LA/PA上分别取得EER 1.93%/1.98%——单一统一模型逼近专用于PA的最优结果（LFCC-CMR，1.34%），同时在LA上仍具竞争力，并在ASVspoof 2021 LA/DF/PA上分别达9.28% / 6.21% / 40.09%。
原文摘要 · Abstract (English)
The dominant trend in voice anti-spoofing fuses self-supervised (SSL) backbones (e.g., WavLM) with handcrafted features, yet such fusion typically lacks transparency in cue-to-layer interactions, and simple concatenation limits cross-modal learning. We propose MOSAIC (Multi-token Oriented Speech Anti-spoofing via Integrated Cross-attention), an interpretable multi-token cross-attention framework that splits a 152-dimensional biophonetic feature vector into six semantic-group query tokens (Praat, phase, LFCC mean/std, sub-band mean/std) and attends them over thirteen mean-std pooled WavLM-Large transformer layers as keys/values. The resulting 6x13 attention matrix visualizes cue-to-layer alignment; a z-score analysis of the per-token activations shows that biophonetic/phase tokens activate more on bona fide speech while spectral/channel tokens activate more on spoofed speech -- yielding per-cue, per-layer attribution that extends prior fusion approaches. Trained jointly with focal loss, a dual LA/PA domain-adversarial classifier, and a bona-fide-only VAE regularizer, MOSAIC attains EER 1.93% / 1.98% on ASVspoof 2019 LA / PA -- a single unified model that approaches the PA-specialized SOTA (LFCC-CMR, 1.34%) while remaining competitive on LA -- and 9.28% / 6.21% / 40.09% on ASVspoof 2021 LA / DF / PA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。