提出BIG-MoE框架，提升多模态人脸反欺骗的泛化能力。
BIG-MoE: Bypass Isolated Gating MoE for Generalized Multimodal Face Anti-Spoofing
- 细粒度专家捕捉细微伪造特征，增强检测精度。
- 隔离门控机制抗输入噪声，提升决策鲁棒性。
- 创新卷积提示旁路设计，改善局部特征感知能力。
在人脸识别安全领域，多模态人脸反欺骗（FAS）对防御呈现攻击至关重要。然而，现有技术受限于模态偏差与不平衡、以及域偏移问题。本文提出一种混合专家（MoE）模型以应对这些挑战。研究发现传统MoE方法在多模态FAS中存在三大局限：(1) 粗粒度专家难以捕捉细微伪造线索；(2) 门控网络易受输入噪声干扰，影响判断；(3) MoE对提示词敏感，传统训练方法易导致过拟合。为此，提出Bypass Isolated Gating MoE（BIG-MoE）框架：(1) 采用细粒度专家，提升对细微伪造信号的检测能力；(2) 设计隔离门控机制，抑制噪声干扰；(3) 引入新型差分卷积提示旁路，将关键局部特征注入门控网络，增强感知能力。在四个基准数据集上的大量实验表明，该方法显著提升了多模态FAS任务的泛化性能。代码已开源：https://github.com/murInJ/BIG-MoE。
原文摘要 · Abstract (English)
In the domain of facial recognition security, multimodal Face Anti-Spoofing (FAS) is essential for countering presentation attacks. However, existing technologies encounter challenges due to modality biases and imbalances, as well as domain shifts. Our research introduces a Mixture of Experts (MoE) model to address these issues effectively. We identified three limitations in traditional MoE approaches to multimodal FAS: (1) Coarse-grained experts' inability to capture nuanced spoofing indicators; (2) Gated networks' susceptibility to input noise affecting decision-making; (3) MoE's sensitivity to prompt tokens leading to overfitting with conventional learning methods. To mitigate these, we propose the Bypass Isolated Gating MoE (BIG-MoE) framework, featuring: (1) Fine-grained experts for enhanced detection of subtle spoofing cues; (2) An isolation gating mechanism to counteract input noise; (3) A novel differential convolutional prompt bypass enriching the gating network with critical local features, thereby improving perceptual capabilities. Extensive experiments on four benchmark datasets demonstrate significant generalization performance improvement in multimodal FAS task. The code is released at https://github.com/murInJ/BIG-MoE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。