通过分析大模型内部表示差异，高效识别生成文本。
RepreGuard: Detecting LLM-Generated Text by Revealing Hidden Representation Patterns
- 利用代理模型提取大模型生成与人类写作的内部激活特征
- 在分布内和分布外场景下平均达94.92%的AUROC性能
- 对不同文本长度和主流攻击保持鲁棒性，适合可信AI应用
检测大语言模型（LLM）生成内容对于防止滥用、构建可信AI系统至关重要。尽管现有检测方法表现良好，但在分布外（OOD）场景下的鲁棒性仍不足。本文假设，相比现有方法使用的特征，大模型内部表示包含更全面、原始的特征，能更有效捕捉并区分大模型生成文本（LGT）与人类写作文本（HWT）之间的统计模式差异。我们在多个大模型上验证了该假设，观察到处理两类文本时神经激活模式存在显著差异。基于此，我们提出RepreGuard——一种高效的基于统计的检测方法。首先，使用代理模型收集LGT与HWT的表示，并提取能更好识别LGT的激活特征；随后通过计算文本表示在此特征方向上的投影得分并与预设阈值比较进行分类。实验结果表明，RepreGuard在分布内和分布外场景下均优于所有基线方法，平均达到94.92% AUROC，且对不同文本长度和主流攻击具有强鲁棒性。数据与代码已公开：https://github.com/NLP2CT/RepreGuard
原文摘要 · Abstract (English)
Detecting content generated by large language models (LLMs) is crucial for preventing misuse and building trustworthy AI systems. Although existing detection methods perform well, their robustness in out-of-distribution (OOD) scenarios is still lacking. In this paper, we hypothesize that, compared to features used by existing detection methods, the internal representations of LLMs contain more comprehensive and raw features that can more effectively capture and distinguish the statistical pattern differences between LLM-generated texts (LGT) and human-written texts (HWT). We validated this hypothesis across different LLMs and observed significant differences in neural activation patterns when processing these two types of texts. Based on this, we propose RepreGuard, an efficient statistics-based detection method. Specifically, we first employ a surrogate model to collect representation of LGT and HWT, and extract the distinct activation feature that can better identify LGT. We can classify the text by calculating the projection score of the text representations along this feature direction and comparing with a precomputed threshold. Experimental results show that RepreGuard outperforms all baselines with average 94.92% AUROC on both in-distribution (ID) and OOD scenarios, while also demonstrating robust resilience to various text sizes and mainstream attacks. Data and code are publicly available at: https://github.com/NLP2CT/RepreGuard
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。