提出首个基于梯度的文本生成对抗工具,可高效绕过AI检测器。
GradEscape: A Gradient-Based Evader Against AI-Generated Text Detectors
- 通过加权嵌入构建可微文本表示,解决文本离散性问题。
- 仅用139M参数,在多个数据集上实现超过现有方法的攻击成功率。
- 支持无模型访问场景,适用于多种语言模型架构和真实检测器。
本文提出GradEscape,首个针对AI生成文本(AIGT)检测器的基于梯度的攻击方法。为克服文本离散导致的不可导问题,引入新型加权嵌入构造策略,使探测器输入可微分。通过从目标检测器获取反馈更新自身参数,实现高成功率攻击且修改极少。针对检测器与逃逸模型间分词器不一致的问题,提出热启动策略,使GradEscape能适配任意语言模型架构。同时结合新型分词推理与模型提取技术,支持仅查询访问下的有效规避。在四个数据集和三种主流语言模型上评估,对比四种前沿逃逸方法,结果表明其在包括110亿参数重写模型在内的多种场景下均表现更优,仅需1.39亿参数。已成功应用于两个真实商业检测器。分析揭示主要漏洞源于训练数据中表达风格差异。提出一种潜在防御策略,并开源代码以推动更鲁棒检测器的发展。
原文摘要 · Abstract (English)
In this paper, we introduce GradEscape, the first gradient-based evader designed to attack AI-generated text (AIGT) detectors. GradEscape overcomes the undifferentiable computation problem, caused by the discrete nature of text, by introducing a novel approach to construct weighted embeddings for the detector input. It then updates the evader model parameters using feedback from victim detectors, achieving high attack success with minimal text modification. To address the issue of tokenizer mismatch between the evader and the detector, we introduce a warm-started evader method, enabling GradEscape to adapt to detectors across any language model architecture. Moreover, we employ novel tokenizer inference and model extraction techniques, facilitating effective evasion even in query-only access. We evaluate GradEscape on four datasets and three widely-used language models, benchmarking it against four state-of-the-art AIGT evaders. Experimental results demonstrate that GradEscape outperforms existing evaders in various scenarios, including with an 11B paraphrase model, while utilizing only 139M parameters. We have successfully applied GradEscape to two real-world commercial AIGT detectors. Our analysis reveals that the primary vulnerability stems from disparity in text expression styles within the training data. We also propose a potential defense strategy to mitigate the threat of AIGT evaders. We open-source our GradEscape for developing more robust AIGT detectors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。