arXiv:2501.19301cs.CLcs.AI2025-01中稿 · EMNLP被引 2

通过棋局结构分析,揭示AI与人类文本在段落间的创作差异。

Beyond checkmate: exploring the creative chokepoints in AI text

  • 以棋局三阶段类比文本结构,分段对比生成特征。
  • 正文段落虽长但语言连贯性更弱,是检测关键点。
  • 人类文本跨段落风格变化更大,适合做识别依据。

大型语言模型(LLMs)的快速发展重塑了文本生成,但也引发滥用担忧,因此检测生成文本(AI文本)日益重要。以往研究多聚焦于识别并‘将死’AI文本,而本研究探索了更少被关注的领域:在引言、正文和结论三个文本段落中,人类与AI文本的细微差异。无论模型在各段落中表现优劣,这些差异都将直接影响其作为创意助手的有效性与边界。我们借鉴国际象棋的开局、中局和残局结构,分析各段落特征,发现尽管AI文本在正文段落与人类写作高度相似,但依赖语言连续性的特征显示出更高偏差,成为最有效的检测区间。此外,人类文本在不同段落间表现出更强的风格多样性,为区分提供了新视角。研究结果为理解人机文本差异提供了新思路,并推动更有效、可解释的检测方法发展。代码已开源:https://github.com/tripto03/chess_inspired_human_ai_text_distinction。

原文摘要 · Abstract (English)

The rapid advancement of Large Language Models (LLMs) has revolutionized text generation but also raised concerns about potential misuse, making detecting LLM-generated text (AI text) increasingly essential. While prior work has focused on identifying AI text and effectively checkmating it, our study investigates a less-explored territory: portraying the nuanced distinctions between human and AI texts across text segments (introduction, body, and conclusion). Whether LLMs excel or falter in incorporating linguistic ingenuity across text segments, the results will critically inform their viability and boundaries as effective creative assistants to humans. Through an analogy with the structure of chess games, comprising opening, middle, and end games, we analyze segment-specific patterns to reveal where the most striking differences lie. Although AI texts closely resemble human writing in the body segment due to its length, deeper analysis shows a higher divergence in features dependent on the continuous flow of language, making it the most informative segment for detection. Additionally, human texts exhibit greater stylistic variation across segments, offering a new lens for distinguishing them from AI. Overall, our findings provide fresh insights into human-AI text differences and pave the way for more effective and interpretable detection strategies. Codes available at https://github.com/tripto03/chess_inspired_human_ai_text_distinction.

文本检测人机差异段落分析风格多样性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。