发现语音对话模型会泄露用户身份，提出两种实时匿名化方案
Privacy-Preserving End-to-End Full-Duplex Speech Dialogue Models
- 在所有层和回合中检测到说话人身份泄露，早期层更严重
- 新方法使误识率提升3.5倍，接近随机水平（41%）
- 可低延迟运行，保留90%以上语音特征，适合实际部署
端到端全双工语音模型通过始终开启的大型语言模型骨干网络处理用户音频，但其隐状态对说话人隐私的影响尚未被研究。依据VoicePrivacy 2024协议，采用懒惰知情攻击者，我们发现SALM-Duplex和Moshi的所有Transformer层均存在显著的说话人身份泄露。分层与分回合分析显示，泄露贯穿所有层级，其中SALM-Duplex在早期层泄漏更强，而Moshi呈现均匀泄漏；链接性在前几轮迅速上升。为此提出两种流式匿名化方案：波形级前端（Anon-W2W）和特征域替换（Anon-W2F）。Anon-W2F相较离散编码器基线使等错误率（EER）提升超3.5倍（11.2% → 41.0%），接近50%随机猜测上限；Anon-W2W在亚秒响应延迟下（FRL < 0.8 s）保持78%-93%基线sBERT性能。
原文摘要 · Abstract (English)
End-to-end full-duplex speech models feed user audio through an always-on LLM backbone, yet the speaker privacy implications of their hidden representations remain unexamined. Following the VoicePrivacy 2024 protocol with a lazy-informed attacker, we show that the hidden states of SALM-Duplex and Moshi leak substantial speaker identity across all transformer layers. Layer-wise and turn-wise analyses reveal that leakage persists across all layers, with SALM-Duplex showing stronger leakage in early layers while Moshi leaks uniformly, and that Linkability rises sharply within the first few turns. We propose two streaming anonymization setups using Stream-Voice-Anon: a waveform-level front-end (Anon-W2W) and a feature-domain replacement (Anon-W2F). Anon-W2F raises EER by over 3.5x relative to the discrete encoder baseline (11.2% to 41.0%), approaching the 50% random-chance ceiling, while Anon-W2W retains 78-93% of baseline sBERT across setups with sub-second response latency (FRL under 0.8 s).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。