arXiv:2605.02718cs.SDcs.MM2026-05

强隐私下训练语音分类模型,防崩溃并保证部署时的音视频一致性。

Private Speech Classification without Collapse: Stabilized DP Training and Offline Distillation

论文配图:Private Speech Classification without Collapse: Stabilized DP Training and Offline Distillation
图 1 · 摘自论文原文
  • 分两阶段训练:先用私有数据训练带多模态信息的教师模型,再离线蒸馏成纯音频学生模型。
  • 在ε≤1强隐私条件下,传统方法易退化为单类预测,准确率虚高但无效。
  • 提出稳定机制,解决隐私训练中的类别失衡与部署模态不匹配问题,适合隐私敏感场景。

我们研究在实际发布约束下的例级私有语音分类:训练可访问特权侧信息,但释放模型必须为纯音频。该设定重要,因语音系统常可在开发中利用更丰富的侧信息,而部署需轻量级、可审计隐私的单模态模型。使用基于 $D_{ ext{priv}}$ 的DP-SGD,在不平衡任务上发现强隐私失败模式(ε≤1),训练可能退化为近乎单类预测,整体准确率会掩盖此问题。因此强调使用宏平均F1、平衡准确率及简单崩溃诊断。此失败在发布设定中尤为严重,因退化的私有教师无法为下游音频仅学生提供有效监督。为此,我们提出两阶段协议:(i) 在 $D_{ ext{priv}}$ 上训练(可能多模态)的DP教师;(ii) 使用固定、录音无关的辅助数据集 $D_{ ext{aux}}$ 离线蒸馏音频仅学生,仅释放学生模型。DP保证仅针对 $D_{ ext{priv}}$;对 $D_{ ext{aux}}$ 不做任何DP声明,且释放学生对 $D_{ ext{priv}}$ 的隐私性由后处理保证。我们将此设定视为四个耦合瓶颈:DP-SGD下的语音诱导优化不稳定、裁剪与噪声导致少数类侵蚀、教师过度依赖部署不可用的特权模态、训练-部署模态错配。通过DP稳定声学前端(DSAF)、小批量自适应有界损失重加权(AW-DP)、特权模态丢弃和离线教师到学生蒸馏予以解决。

原文摘要 · Abstract (English)

We study example-level private supervised speech classification under a practical release constraint: training may access privileged side information, but the released model must be audio-only. This setting is important because speech systems can often exploit richer side information during development, whereas deployment and release require a lightweight unimodal model with auditable privacy guarantees. Using DP-SGD on the private dataset $D_{\text{priv}}$, we identify a strong-privacy failure mode ($ε\le 1$) on imbalanced tasks, where training may collapse to a near single-class predictor, a phenomenon that overall accuracy can obscure. We therefore emphasize Macro-F1, balanced accuracy, and a simple collapse diagnostic. This failure is especially problematic in our release setting because a collapsed private teacher cannot provide useful supervision for the downstream audio-only student. To address this setting under strong privacy, we propose a two-stage protocol: (i) train a (possibly multimodal) DP teacher on $D_{\text{priv}}$, and (ii) distill an audio-only student on a fixed, recording-disjoint auxiliary dataset $D_{\text{aux}}$ using one-shot offline teacher probability outputs, releasing only the student. The DP guarantee applies only to $D_{\text{priv}}$; we make no DP claim for $D_{\text{aux}}$, and privacy of the released student with respect to $D_{\text{priv}}$ follows by post-processing. We frame this setting as involving four coupled bottlenecks: speech-induced optimization instability under DP-SGD, minority-class erosion under clipping and noise, teacher over-reliance on privileged modalities unavailable at deployment, and train--deploy modality mismatch. We address them with a DP-stabilizing acoustic front-end (DSAF), minibatch-adaptive bounded loss reweighting (AW-DP), privileged-modality dropout, and offline teacher-to-student distillation.

语音分类差分隐私知识蒸馏隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。