改进视频模型读出机制,让冻结的视频骨干网络更擅长检测AI生成视频。
Rethinking the Readout: Unlocking Video Backbones for AI-Generated Video Detection

- 设计双流结构,仅替换聚合层,捕捉帧间局部动态变化。
- 在AIGVDBench上达95.28% AUC,显著超越图像模型基准。
- 轻量级模块可通用,无需微调即可激活视频骨干的时序潜力。
AI生成视频通常包含由帧间不一致性引发的细微时序伪影,而非单帧内问题。因此,依赖视频预训练骨干网络的检测器应更具优势。然而,实践中采用标准全局读出的视频骨干往往不如强图像预训练探针表现优异。我们归因于读出层过度时空聚合:视频骨干常将每帧压缩为单一全局描述符,抑制了局部补丁级时序动态并丢失补丁间关系,而这些正是检测AI生成视频最可靠的线索。为此,我们提出速度门控补丁速度分析(V-PVP),一种仅替换聚合层的轻量级读出模块,新增约0.5M可训练参数。该模块作为通用插件,在多种视频骨干中均提升性能,适用于端到端微调与线性探测场景。方法在保持骨干冻结时达到95.28% AUC,证明仅更换聚合层即可重激活冻结视频骨干的时序潜力,恢复其优势。代码已公开。
原文摘要 · Abstract (English)
AI-generated videos (AIGVs) typically contain subtle temporal artifacts that arise from inter-frame inconsistencies rather than within individual frames. A detector that captures such artifacts should therefore benefit from video pretrained backbones over image only ones. In practice, however, video backbones with standard global readouts often fail to outperform strong image pretrained probes on AIGV benchmarks. We attribute this gap to excessive spatiotemporal aggregation in the readout. Video pretrained backbones tend to compress each frame into a single global descriptor. This compression suppresses local patch level temporal dynamics and discards inter patch relations, which are precisely the cues that AIGV detection most reliably depends on. Based on this, we propose Velocity Gated Patch Velocity Profiling (V-PVP), a lightweight readout that replaces only the aggregation layer with two parallel streams over the patch velocity field, adding only about $0.5$M trainable parameters. V-PVP serves as a general plug-and-play module that consistently improves performance across diverse video backbones under both end-to-end fine-tuning and linear probing settings. Our method reaches \textbf{95.28} AUC on AIGVDBench while keeping the backbone fully frozen. The results show that simply replacing the aggregation layer reactivates the temporal potential of frozen video backbones, restoring their advantage on AIGV detection. Code is available at https://anonymous.4open.science/r/PVP-81B3/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。