arXiv:2603.01433cs.CV2026-03被引 2

首个零样本文书伪造检测基准,揭示现有方法因阈值失准而失效。

DOCFORGE-BENCH: A Comprehensive 0-shot Benchmark for Document Forgery Detection and Analysis

  • 用预训练模型零样本评估14种方法,模拟真实无标注数据场景。
  • 检测准确率(AUC)中等但像素精确率(F1)接近零,差距达10倍。
  • 只需少量样本调阈值即可大幅提升性能,适合实际部署者参考。

我们提出DOCFORGE-BENCH,首个统一的零样本文书伪造检测基准,评估14种方法在8个数据集上的表现,涵盖文本篡改、收据伪造和身份文件篡改。与依赖微调的评测如ForensicHub不同,DOCFORGE-BENCH使用各方法发布的预训练权重且不进行领域适配,更贴近实际部署场景——从业者缺乏标注数据。核心发现是:在单阈值协议下,方法虽有中等像素级AUC(≥0.76),但像素级F1近乎为零。这一差距并非分类能力不足,而是分数分布偏移所致:篡改区域仅占文档图像0.27%-4.17%像素,远低于自然图像基准,导致标准tau=0.5阈值严重失准。基于人工标注的最优阈值(Oracle-F1)比固定阈值高出2-10倍,证实瓶颈在于校准而非表征。控制实验表明,在N=10张域内图像上调整阈值,可恢复39%-55%的性能差距,证明阈值自适应(非重训练)是实用部署的关键缺失步骤。总体而言,当前所有方法均无法在多样化文书上可靠运行,表明该问题仍未解决。此外,所有8个数据集均早于生成式AI编辑时代;覆盖扩散模型与大语言模型生成的伪造文书,构成现代攻击面的重要空白。

原文摘要 · Abstract (English)

We present DOCFORGE-BENCH, the first unified zero-shot benchmark for document forgery detection, evaluating 14 methods across eight datasets spanning text tampering, receipt forgery, and identity document manipulation. Unlike fine-tuning-oriented evaluations such as ForensicHub [Du et al., 2025], DOCFORGE-BENCH applies all methods with their published pretrained weights and no domain adaptation -- a deliberate design choice that reflects the realistic deployment scenario where practitioners lack labeled document training data. Our central finding is a pervasive calibration failure invisible under single-threshold protocols: methods achieve moderate Pixel-AUC (>=0.76) yet near-zero Pixel-F1. This AUC-F1 gap is not a discrimination failure but a score-distribution shift: tampered regions occupy only 0.27-4.17% of pixels in document images -- an order of magnitude less than in natural image benchmarks -- making the standard tau=0.5 threshold catastrophically miscalibrated. Oracle-F1 is 2-10x higher than fixed-threshold Pixel-F1, confirming that calibration, not representation, is the bottleneck. A controlled calibration experiment validates this: adapting a single threshold on N=10 domain images recovers 39-55% of the Oracle-F1 gap, demonstrating that threshold adaptation -- not retraining -- is the key missing step for practical deployment. Overall, no evaluated method works reliably out-of-the-box on diverse document types, underscoring that document forgery detection remains an unsolved problem. We further note that all eight datasets predate the era of generative AI editing; benchmarks covering diffusion- and LLM-based document forgeries represent a critical open gap on the modern attack surface.

文书伪造零样本阈值校准AI检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。