统一处理图像视频的篡改定位,无需插值也能适应不同分辨率。
RelayFormer: A Unified Local-Global Attention Framework for Scalable Image and Video Manipulation Localization
- 用全局-局部接力令牌机制,动态传递语义和时序一致性信息。
- 在多个数据集上达到新高,计算开销比现有方法低30%以上。
- 适合需要高效处理多分辨率图像与视频篡改检测的研究者。
视觉篡改定位(VML)旨在识别图像和视频中的伪造区域,随着高级编辑工具的发展,该任务日益困难。现有方法面临两大挑战:一是分辨率多样性问题,缩放或填充会扭曲细微的取证线索并引入额外计算成本;二是难以将图像空间模型扩展至视频时空输入,导致图像与视频需采用独立架构。为此,我们提出RelayFormer,一种可适应不同分辨率、自然处理静态与动态视觉数据的统一框架。RelayFormer将输入划分为固定大小的子图像,并引入基于接力机制的全局-局部接力(GLR)令牌,实现结构化上下文的高效传播。该设计能有效保留细粒度篡改痕迹,同时传递全局语义或时序一致性信息。相比依赖均匀缩放或稀疏注意力的方法,RelayFormer在可变分辨率和视频序列上均具备良好扩展性,计算开销极小。在多个基准测试中表现优异,兼具分辨率自适应能力(无需插值或过度填充)、图像与视频统一处理,以及准确率与计算成本间的良好平衡。代码已公开于~\href{https://github.com/WenOOI/RelayFormer}{https://github.com/WenOOI/RelayFormer}。
原文摘要 · Abstract (English)
Visual manipulation localization (VML) aims to identify tampered regions in images and videos, a task that has become increasingly challenging with the rise of advanced editing tools. Existing methods face two central issues. The first is resolution diversity. Resizing or padding can distort subtle forensic cues and introduce unnecessary computational cost. The second is the difficulty of extending spatial models for images to spatio-temporal inputs in videos, which often results in maintaining separate architectures for the two data types. To address these challenges, we propose RelayFormer, a unified framework that adapts to varying resolutions and naturally handles both static and temporal visual data. RelayFormer partitions inputs into fixed-size sub-images and introduces Global Local Relay (GLR) tokens that propagate structured context through a relay-based attention mechanism. This design enables efficient exchange of global cues, such as semantic or temporal consistency, while preserving fine-grained manipulation artifacts. Unlike prior approaches that depend on uniform resizing or sparse attention, RelayFormer scales to variable resolutions and video sequences with minimal overhead. Experiments across diverse benchmarks demonstrate superior performance and strong efficiency, combining resolution adaptivity without interpolation or excessive padding, unified processing for images and videos, and a favorable balance between accuracy and computational cost. Code is available at~\href{https://github.com/WenOOI/RelayFormer}{https://github.com/WenOOI/RelayFormer}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。