用学生t分布改进检测模型,更准识别对抗性文本。
T-Detect: Tail-Aware Statistical Normalization for Robust Detection of Adversarial Machine-Generated Text
- 改用学生t分布替代高斯分布,更好处理对抗文本的长尾特征。
- 在RAID和HART数据集上,最高提升3.9%的AUROC性能。
- 适合需要高鲁棒性的文本生成检测场景,如内容安全审核。
大型语言模型生成的文本日益流畅,给机器生成文本检测带来挑战,尤其是经过对抗性扰动(如改写)的文本。现有零样本检测器常采用高斯分布作为统计度量,但面对对抗性或非母语英文文本的重尾统计特征时表现不佳。本文提出T-Detect,一种重新设计的基于曲率的检测方法,核心创新是将标准高斯归一化替换为源自学生t分布的重尾差异分数。该方法基于实证观察:对抗文本具有显著尖峰厚尾特征,传统统计假设不适用。T-Detect通过将段落的对数似然相对于t分布的期望矩进行归一化,实现对统计异常值的更强鲁棒性。我们在具有挑战性的RAID对抗文本检测基准和全面的HART数据集上验证该方法。实验表明,T-Detect持续优于强基线,目标领域中最高提升3.9%的AUROC。集成至二维检测框架(CT)后,在RAID的Books领域达到0.926的AUROC,达当前最优水平。贡献包括一个理论支持的新统计基础、经消融验证的鲁棒方法,以及在对抗条件下的全面性能分析。代码已开源于https://github.com/ResearAI/t-detect。
原文摘要 · Abstract (English)
Large language models (LLMs) have shown the capability to generate fluent and logical content, presenting significant challenges to machine-generated text detection, particularly text polished by adversarial perturbations such as paraphrasing. Current zero-shot detectors often employ Gaussian distributions as statistical measure for computing detection thresholds, which falters when confronted with the heavy-tailed statistical artifacts characteristic of adversarial or non-native English texts. In this paper, we introduce T-Detect, a novel detection method that fundamentally redesigns the curvature-based detectors. Our primary innovation is the replacement of standard Gaussian normalization with a heavy-tailed discrepancy score derived from the Student's t-distribution. This approach is theoretically grounded in the empirical observation that adversarial texts exhibit significant leptokurtosis, rendering traditional statistical assumptions inadequate. T-Detect computes a detection score by normalizing the log-likelihood of a passage against the expected moments of a t-distribution, providing superior resilience to statistical outliers. We validate our approach on the challenging RAID benchmark for adversarial text and the comprehensive HART dataset. Experiments show that T-Detect provides a consistent performance uplift over strong baselines, improving AUROC by up to 3.9\% in targeted domains. When integrated into a two-dimensional detection framework (CT), our method achieves state-of-the-art performance, with an AUROC of 0.926 on the Books domain of RAID. Our contributions are a new, theoretically-justified statistical foundation for text detection, an ablation-validated method that demonstrates superior robustness, and a comprehensive analysis of its performance under adversarial conditions. Ours code are released at https://github.com/ResearAI/t-detect.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。