用可变形动态和接力令牌定位视频音频中的伪造片段
DeformTrace: A Deformable State Space Model with Relay Tokens for Temporal Forgery Localization
- 引入可变感受野与接力令牌，提升时序定位精度
- 在仅25%参数量下达到顶尖性能，推理速度更快
- 适合需要高效高精度伪造检测的应用场景
时间伪造定位（TFL）旨在精准识别视频和音频中的篡改片段，为安全与取证提供强可解释性。尽管近期状态空间模型（SSMs）在精确时序推理方面展现潜力，但其在TFL中仍受限于边界模糊、伪造稀疏以及长程建模能力不足。本文提出DeformTrace，通过可变形动态与接力机制增强SSMs以应对这些挑战。具体而言，可变形自态空间模型（DS-SSM）引入动态感受野，实现精准时序定位；并集成接力令牌机制，增强时序推理能力并缓解长程衰减问题。此外，可变形交叉态空间模型（DC-SSM）将全局状态空间划分为查询相关的子空间，减少非伪造信息累积，提升对稀疏伪造的敏感性。上述组件融合于结合Transformer全局建模与SSM高效性的混合架构中。大量实验表明，DeformTrace在参数更少、推理更快的同时，取得当前最优性能，并具备更强鲁棒性。
原文摘要 · Abstract (English)
Temporal Forgery Localization (TFL) aims to precisely identify manipulated segments in video and audio, offering strong interpretability for security and forensics. While recent State Space Models (SSMs) show promise in precise temporal reasoning, their use in TFL is hindered by ambiguous boundaries, sparse forgeries, and limited long-range modeling. We propose DeformTrace, which enhances SSMs with deformable dynamics and relay mechanisms to address these challenges. Specifically, Deformable Self-SSM (DS-SSM) introduces dynamic receptive fields into SSMs for precise temporal localization. To further enhance its capacity for temporal reasoning and mitigate long-range decay, a Relay Token Mechanism is integrated into DS-SSM. Besides, Deformable Cross-SSM (DC-SSM) partitions the global state space into query-specific subspaces, reducing non-forgery information accumulation and boosting sensitivity to sparse forgeries. These components are integrated into a hybrid architecture that combines the global modeling of Transformers with the efficiency of SSMs. Extensive experiments show that DeformTrace achieves state-of-the-art performance with fewer parameters, faster inference, and stronger robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。