arXiv:2511.00416cs.CLcs.AI2025-11被引 4

测试AI文本检测器在改写攻击下的失效情况,发现其难以识别伪装成人类写作的生成文本。

PADBen: A Comprehensive Benchmark for Evaluating AI Text Detectors Against Paraphrase Attacks

  • 构建五类文本层级,模拟从原始生成到深度伪装的改写过程。
  • 11个主流检测器在作者身份混淆场景下准确率不足50%。
  • 揭示现有检测方法对中间清洗区无效,适合安全与内容审核研究者参考。

尽管AI生成文本(AIGT)检测器在直接大模型输出上准确率超过90%,但在迭代改写内容面前会彻底失效。我们分析了为何迭代改写文本——本身由AI生成——能逃避针对AIGT识别的检测系统。通过内在机制分析,发现迭代改写形成一个中间清洗区域:语义偏移但保留生成模式,导致两类攻击:改写人类文本(作者身份混淆)和改写大模型生成文本(抄袭规避)。为应对这些漏洞,我们提出PADBen,首个系统评估检测器在两种改写攻击场景下鲁棒性的基准。PADBen包含五类文本分类,覆盖从原始内容到深度伪装的完整轨迹,并设置五项渐进式检测任务,涵盖句对与单句挑战。我们评估了11个前沿检测器,发现关键不对称性:检测器能有效识别抄袭规避问题,但在作者身份混淆场景中失败。研究结果表明,当前检测方法无法有效处理中间清洗区域,亟需超越现有语义与风格判别的架构革新。代码详见 https://github.com/JonathanZha47/PadBen-Paraphrase-Attack-Benchmark。

原文摘要 · Abstract (English)

While AI-generated text (AIGT) detectors achieve over 90\% accuracy on direct LLM outputs, they fail catastrophically against iteratively-paraphrased content. We investigate why iteratively-paraphrased text -- itself AI-generated -- evades detection systems designed for AIGT identification. Through intrinsic mechanism analysis, we reveal that iterative paraphrasing creates an intermediate laundering region characterized by semantic displacement with preserved generation patterns, which brings up two attack categories: paraphrasing human-authored text (authorship obfuscation) and paraphrasing LLM-generated text (plagiarism evasion). To address these vulnerabilities, we introduce PADBen, the first benchmark systematically evaluating detector robustness against both paraphrase attack scenarios. PADBen comprises a five-type text taxonomy capturing the full trajectory from original content to deeply laundered text, and five progressive detection tasks across sentence-pair and single-sentence challenges. We evaluate 11 state-of-the-art detectors, revealing critical asymmetry: detectors successfully identify the plagiarism evasion problem but fail for the case of authorship obfuscation. Our findings demonstrate that current detection approaches cannot effectively handle the intermediate laundering region, necessitating fundamental advances in detection architectures beyond existing semantic and stylistic discrimination methods. For detailed code implementation, please see https://github.com/JonathanZha47/PadBen-Paraphrase-Attack-Benchmark.

AI检测改写攻击文本安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。