arXiv:2410.18371cs.SDcs.AI2024-10被引 13

用随机文本检测音频模型隐私泄露,无需真实音频或影子模型。

Gibberish is All You Need for Membership Inference Detection in Contrastive Language-Audio Pretraining

  • 用随机文本生成异常特征,训练无监督检测器
  • 在多个数据集上优于仅用文本的基线方法
  • 适合保护语音隐私的场景,尤其无音频可用时

音频可能泄露个人身份信息,尤其当与文本数据结合时。因此,有必要开发工具来检测对比语言-音频预训练(CLAP)中的隐私泄露。现有成员推理攻击(MIA)需要输入音频,存在声纹暴露风险且需昂贵的影子模型。我们首先提出PRMID,一种基于CLAP输出概率排序的成员推理检测器,无需训练影子模型,但仍需个体的音频和文本。为克服这些限制,我们进一步提出USMID,一种仅使用文本的单模态说话人级成员推理检测器:通过随机生成明显不在训练集中的文本杂音,利用CLAP提取特征向量并训练异常检测器。推理时,将测试文本特征输入检测器,判断说话人是否在训练集中(异常)或否(正常)。若可获得真实音频,USMID可进一步提升检测效果。在多种CLAP模型架构和数据集上的大量实验表明,USMID在仅用文本的情况下优于基线方法。

原文摘要 · Abstract (English)

Audio can disclose PII, particularly when combined with related text data. Therefore, it is essential to develop tools to detect privacy leakage in Contrastive Language-Audio Pretraining(CLAP). Existing MIAs need audio as input, risking exposure of voiceprint and requiring costly shadow models. We first propose PRMID, a membership inference detector based probability ranking given by CLAP, which does not require training shadow models but still requires both audio and text of the individual as input. To address these limitations, we then propose USMID, a textual unimodal speaker-level membership inference detector, querying the target model using only text data. We randomly generate textual gibberish that are clearly not in training dataset. Then we extract feature vectors from these texts using the CLAP model and train a set of anomaly detectors on them. During inference, the feature vector of each test text is input into the anomaly detector to determine if the speaker is in the training set (anomalous) or not (normal). If available, USMID can further enhance detection by integrating real audio of the tested speaker. Extensive experiments on various CLAP model architectures and datasets demonstrate that USMID outperforms baseline methods using only text data.

隐私检测语音安全无监督学习文本攻击

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。