一个模型搞定人脸伪造检测四类任务,效率更高更准确。
OmniFD: A Unified Model for Versatile Face Forgery Detection
- 用统一框架同时处理图像视频分类、定位等四类任务
- 融合图像数据使视频分类准确率提升4.63%
- 参数减少63%,训练时间缩短50%,适合实际应用
人脸伪造检测包含多个关键任务:识别伪造图像与视频、定位篡改区域及时间片段。现有方法多采用独立的专用模型,造成计算冗余并忽略任务间关联。我们提出OmniFD,一个统一框架,在单一模型中联合解决四类核心任务:图像与视频分类、空间定位、时间定位。其架构包含三个核心组件:(1) 共享Swin Transformer编码器,从图像和视频输入中提取统一的4D时空表示;(2) 带可学习查询的跨任务交互模块,通过注意力机制动态捕捉任务间依赖;(3) 轻量解码头,将优化表示转化为各类任务预测。大量实验表明,相比专用模型,OmniFD在多基准测试中表现更优。其统一设计利用多任务学习捕获泛化表征,尤其实现细粒度知识迁移——引入图像数据后,视频分类准确率提升4.63%。通过将图像、视频与四类任务统一于一个框架,OmniFD在多样基准上实现更优性能,兼具高效率与可扩展性,例如参数减少63%,训练时间降低50%。该方案为真实场景下的全面人脸伪造检测提供了实用且通用的解决方案。源代码已公开于 https://github.com/haotianll/OmniFD。
原文摘要 · Abstract (English)
Face forgery detection encompasses multiple critical tasks, including identifying forged images and videos and localizing manipulated regions and temporal segments. Current approaches typically employ task-specific models with independent architectures, leading to computational redundancy and ignoring potential correlations across related tasks. We introduce OmniFD, a unified framework that jointly addresses four core face forgery detection tasks within a single model, i.e., image and video classification, spatial localization, and temporal localization. Our architecture consists of three principal components: (1) a shared Swin Transformer encoder that extracts unified 4D spatiotemporal representations from both images and video inputs, (2) a cross-task interaction module with learnable queries that dynamically captures inter-task dependencies through attention-based reasoning, and (3) lightweight decoding heads that transform refined representations into corresponding predictions for all FFD tasks. Extensive experiments demonstrate OmniFD's advantage over task-specific models. Its unified design leverages multi-task learning to capture generalized representations across tasks, especially enabling fine-grained knowledge transfer that facilitates other tasks. For example, video classification accuracy improves by 4.63% when image data are incorporated. Furthermore, by unifying images, videos and the four tasks within one framework, OmniFD achieves superior performance across diverse benchmarks with high efficiency and scalability, e.g., reducing 63% model parameters and 50% training time. It establishes a practical and generalizable solution for comprehensive face forgery detection in real-world applications. The source code is made available at https://github.com/haotianll/OmniFD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。