arXiv:2502.04528cs.CLcs.LG2025-02被引 3

针对AI生成文本检测中阈值僵化问题,提出分组自适应优化方法

Group-Adaptive Threshold Optimization for Robust AI-Generated Text Detection

  • 按文本长度、风格等属性分组,为每组学习专属阈值
  • 在九个检测器上使最大差异降低27.4%,准确率仅下降0.005%
  • 适合关注公平性与鲁棒性的内容审核系统开发者

大语言模型的发展使得区分人类写作与机器生成文本愈发困难。现有检测器通常采用固定全局阈值(如θ=0.5)进行分类,但单一阈值难以应对不同子群体的分布差异。例如,固定阈值在短文本上产生更多误报,在长文本中对神经质写作风格过度识别,导致某些群体被不公平对待。本文提出FairOPT算法,基于文本长度、写作风格等属性对数据分组,并为各组优化决策阈值以减少偏差。在三个异构数据集和九个检测器上验证,FairOPT显著降低最大差异,整体偏差下降27.4%,同时准确率仅微降0.005%。该框架通过后处理提升AI生成内容检测的鲁棒性。代码与数据已公开。

原文摘要 · Abstract (English)

The advancement of large language models (LLMs) has made it difficult to differentiate human-written text from AI-generated text. Several AI-text detectors have been developed in response, which typically utilize a fixed global threshold (e.g., $θ= 0.5$) to classify machine-generated text. However, one universal threshold could fail to account for distributional variations by subgroups. For example, when using a fixed threshold, detectors make more false positive errors on shorter human-written text, and more positive classifications of neurotic writing styles among long texts. These discrepancies can lead to misclassifications that disproportionately affect certain groups. We address this critical limitation by introducing FairOPT, an algorithm for group-specific threshold optimization for probabilistic AI-text detectors. We partitioned data into subgroups based on attributes (e.g., text length and writing style) and implemented FairOPT to learn decision thresholds for each group to reduce discrepancy. FairOPT showed notable discrepancy mitigation across nine detectors and three heterogeneous datasets, and the remarkable mitigation of the minimax problem by decreasing overall discrepancy 27.4% across five metrics while minimally sacrificing accuracy by 0.005%. Our framework paves the way for more robust classification in AI-generated content detection via post-processing. We release our data, code, and project information at URL.

文本检测公平性阈值优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。