用自回归生成方式精准定位图像篡改,无需中间分割步骤。
ForensicsTok: Forensics-Guided Tokenized Modeling for Image Tampering Localization

- 将篡改定位转为序列生成任务,直接输出带空间信息的标记序列。
- 在6个数据集上优于现有大模型基线,对干扰更鲁棒。
- 融合多尺度伪造特征,弥补通用大模型的领域缺陷,适合专业检测场景。
多模态大语言模型(MLLM)在取证任务中具备强大推理能力,但现有方法依赖外部分割解码器,常导致定位效果不佳。由于拼接式流程在反向传播中引入信息瓶颈,空间信号被削弱,且受限于分割器的语义先验。为此,我们提出 ForensicsTok,将图像篡改定位重构为自回归序列生成任务,直接生成具有空间定位性的标记序列,实现无中间监督的精确掩码预测。具体地,提出令牌溅射解码器(TSD),通过码本感知的码平滑将标记映射为二值掩码,缓解确定性解码器带来的尖锐梯度问题。此外,设计分层专家融合(HEF)模块,注入来自取证专家模型的多尺度特征,以捕捉多样化的篡改线索。该统一架构有效弥补了标准 MLLM 在取证先验上的不足。在六个基准上的大量实验表明,ForensicsTok 显著优于现有 MLLM 基线,并略胜于强健的取证专家基线,同时对扰动表现出更强鲁棒性。
原文摘要 · Abstract (English)
Multi-modal Large Language Models (MLLMs) offer powerful reasoning for forensic tasks, yet existing approaches utilizing exogenous segmentation decoders often suffer from suboptimal localization. The reliance on stitched pipelines introduces information bottlenecks during backpropagation, which dilutes spatial signals and is limited by semantic priors of the segmentor. To address these limitations, we propose ForensicsTok, which reformulates image manipulation localization as an autoregressive sequence generation task. ForensicsTok directly generates spatially grounded token sequences, enabling precise mask prediction without intermediary supervision. Specifically, we introduce a Token Splatting Decoder (TSD) to map tokens to binary masks via codebook-aware code smoothing, which mitigates sharp gradients from deterministic detokenizers. Furthermore, to capture diverse tampering clues, we propose a Hierarchical Expert Fusion (HEF) module that injects multi-scale features from a forensic expert model. This unified architecture effectively compensates for the lack of forensic priors in standard MLLMs. Extensive experiments on six benchmarks show that ForensicsTok substantially improves over existing MLLM-based baselines and slightly improves over strong forensic expert baselines, while exhibiting stronger robustness to perturbations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。