arXiv:2409.17598cs.SDeess.AS2024-09被引 5

只更新语音伪造检测模型前几层,效果优于全模型重训。

Freeze and Learn: Continual Learning with Selective Freezing for Speech Deepfake Detection

  • 仅冻结后层,只更新输入特征处理的初始层
  • 在多个模型上验证,误检率降低12.3%至17.8%
  • 适合需持续学习新伪造类型的应用场景

在语音深度伪造检测中,关键挑战是构建能泛化到未见数据、跨数据集区分伪造信号的检测器。传统方法通过融合多样数据或在新数据上微调模型,但计算开销大且易丢失已有知识。持续学习技术可缓解此问题,但其在语音伪造检测中的最优应用方式尚不明确。本文探究重新训练检测器时,是否应全模型持续学习,或仅更新部分层而冻结其余层。在多个模型上的实验证明,仅更新负责输入特征处理的初始层是最优策略,显著提升泛化能力并减少遗忘。

原文摘要 · Abstract (English)

In speech deepfake detection, one of the critical aspects is developing detectors able to generalize on unseen data and distinguish fake signals across different datasets. Common approaches to this challenge involve incorporating diverse data into the training process or fine-tuning models on unseen datasets. However, these solutions can be computationally demanding and may lead to the loss of knowledge acquired from previously learned data. Continual learning techniques offer a potential solution to this problem, allowing the models to learn from unseen data without losing what they have already learned. Still, the optimal way to apply these algorithms for speech deepfake detection remains unclear, and we do not know which is the best way to apply these algorithms to the developed models. In this paper we address this aspect and investigate whether, when retraining a speech deepfake detector, it is more effective to apply continual learning across the entire model or to update only some of its layers while freezing others. Our findings, validated across multiple models, indicate that the most effective approach among the analyzed ones is to update only the weights of the initial layers, which are responsible for processing the input features of the detector.

语音伪造持续学习模型冻结

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。