用逆向提示词检测大模型文本，效果强且结果可解释。
IPAD: Inverse Prompt for AI Detection - A Robust and Interpretable LLM-Generated Text Detector
- 通过反推生成文本的提示词来判断是否为大模型生成。
- 在分布内、分布外及攻击数据上分别提升9.05%、12.93%、5.48%性能。
- 能提供可查看的决策依据，适合需要可信检测的场景。
大语言模型已达到人类水平的文本生成能力，使区分人工撰写与模型生成文本变得困难，加剧了滥用风险，亟需可靠检测工具。然而现有检测器在分布外（OOD）和受攻击数据上表现差，且缺乏可解释性证据，影响可靠性。为此，我们提出IPAD（逆向提示词用于AI检测），包含一个提示词反演模块，用于识别可能生成输入文本的提示词，以及两个判别器，评估输入文本与预测提示词的匹配概率。实证表明，IPAD在分布内数据上平均召回率优于最强基线9.05%，在分布外数据上AUROC提升12.93%，在受攻击数据上提升5.48%。该方法在结构化数据集上也表现稳健。可解释性评估显示，用户可直接审查决策依据，增强了检测结果的可信度。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have attained human-level fluency in text generation, which complicates the distinguishing between human-written and LLM-generated texts. This increases the risk of misuse and highlights the need for reliable detectors. Yet, existing detectors exhibit poor robustness on out-of-distribution (OOD) data and attacked data, which is critical for real-world scenarios. Also, they struggle to provide interpretable evidence to support their decisions, thus undermining the reliability. In light of these challenges, we propose IPAD (Inverse Prompt for AI Detection), a novel framework consisting of a Prompt Inverter that identifies predicted prompts that could have generated the input text, and two Distinguishers that examine the probability that the input texts align with the predicted prompts. Empirical evaluations demonstrate that IPAD outperforms the strongest baselines by 9.05% (Average Recall) on in-distribution data, 12.93% (AUROC) on out-of-distribution data, and 5.48% (AUROC) on attacked data. IPAD also performs robustly on structured datasets. Furthermore, an interpretability assessment is conducted to illustrate that IPAD enhances the AI detection trustworthiness by allowing users to directly examine the decision-making evidence, which provides interpretable support for its state-of-the-art detection results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。