解决多模态人脸反欺骗中的模态与域对齐问题,提升跨场景泛化能力。
DADM: Dual Alignment of Domain and Modality for Face Anti-spoofing
- 设计互信息对齐模块,自适应增强有效模态、抑制无效模态。
- 提出双对齐优化方法,缓解模态间域偏移,提升跨域鲁棒性。
- 在四个挑战性协议上达领先性能,适合实际部署的多模态反欺骗系统。
随着RGB、深度、红外等多种传感器模态的普及及多模态学习的成功,多模态人脸反欺骗(FAS)成为研究热点。其核心思想是融合多模态信息可揭示更深层伪造痕迹。然而,该方法面临两大对齐失准问题:(1) 模态内域对齐失准,不同攻击类型下各模态有效性不一(如深度模态对3D面具攻击无效);(2) 模态间域对齐失准,新增模态加剧域偏移,掩盖互补优势。为此,本文提出双对齐域与模态(DADM)框架:基于互信息的模态对齐模块自适应调整模态权重;采用双对齐优化策略,同时对齐子域超平面与模态角度边界,以缩小域间差异。在四个具有挑战性的测试协议上验证,DADM实现最先进性能,展现出强跨域泛化能力。代码即将开源。
原文摘要 · Abstract (English)
With the availability of diverse sensor modalities (i.e., RGB, Depth, Infrared) and the success of multi-modal learning, multi-modal face anti-spoofing (FAS) has emerged as a prominent research focus. The intuition behind it is that leveraging multiple modalities can uncover more intrinsic spoofing traces. However, this approach presents more risk of misalignment. We identify two main types of misalignment: (1) \textbf{Intra-domain modality misalignment}, where the importance of each modality varies across different attacks. For instance, certain modalities (e.g., Depth) may be non-defensive against specific attacks (e.g., 3D mask), indicating that each modality has unique strengths and weaknesses in countering particular attacks. Consequently, simple fusion strategies may fall short. (2) \textbf{Inter-domain modality misalignment}, where the introduction of additional modalities exacerbates domain shifts, potentially overshadowing the benefits of complementary fusion. To tackle (1), we propose a alignment module between modalities based on mutual information, which adaptively enhances favorable modalities while suppressing unfavorable ones. To address (2), we employ a dual alignment optimization method that aligns both sub-domain hyperplanes and modality angle margins, thereby mitigating domain gaps. Our method, dubbed \textbf{D}ual \textbf{A}lignment of \textbf{D}omain and \textbf{M}odality (DADM), achieves state-of-the-art performance in extensive experiments across four challenging protocols demonstrating its robustness in multi-modal domain generalization scenarios. The codes will be released soon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。