arXiv:2502.11336cs.CL2025-02ACL被引 4

ExaGPT通过比对文本相似片段,让机器判断更像人一样可解释。

ExaGPT: Example-Based Machine-Generated Text Detection for Human Interpretability

  • 基于人类判断习惯,比对文本与真人/模型生成样本的相似段落
  • 在1%误报率下,准确率最高提升37.0点,超越现有方法
  • 提供具体相似段落作为证据,帮助用户理解判断依据

检测大语言模型(LLM)生成文本可能因错误决策而造成严重后果,例如损害学生学术尊严。因此,检测结果必须具备可解释性,帮助用户判断预测可靠性。当人类判断文本来源时,会自然关注其与真人或模型生成文本的相似段落。然而,现有可解释检测方法未遵循人类决策过程,难以提供易懂的证据。为此,我们提出ExaGPT,一种基于人类决策机制的可解释检测方法。该方法通过检查待测文本在数据存储中与真人写作或模型生成文本的相似段落数量,判断其来源。ExaGPT能为每个文本段落提供支持判断的相似段落示例。人类评估表明,提供相似段落示例显著提升用户对判断正确性的认知。此外,在四个领域和三种生成器上的实验显示,ExaGPT在1%误报率下准确率最高提升37.0点,远超先前可解释检测方法。

原文摘要 · Abstract (English)

Detecting texts generated by Large Language Models (LLMs) could cause grave mistakes due to incorrect decisions, such as undermining students' academic dignity. LLM text detection thus needs to ensure the interpretability of the decision, which can help users judge how reliably correct its prediction is. When humans verify whether a text is human-written or LLM-generated, they intuitively investigate which of them it shares more similar spans with. However, existing interpretable detectors are not aligned with the human decision-making process and fail to offer evidence that users easily understand. To bridge this gap, we introduce ExaGPT, an interpretable detection approach grounded in the human decision-making process for verifying the origin of a text. ExaGPT identifies a text by checking whether it shares more similar spans with human-written vs. with LLM-generated texts from a datastore. This approach can provide similar span examples that contribute to the decision for each span in the text as evidence. Our human evaluation demonstrates that providing similar span examples contributes more effectively to judging the correctness of the decision than existing interpretable methods. Moreover, extensive experiments in four domains and three generators show that ExaGPT massively outperforms prior interpretable detectors by up to +37.0 points of accuracy at a false positive rate of 1%.

文本检测可解释性LLM安全证据生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。