分析LLM生成文本如何影响传统检索模型的性能表现
How Do LLM-Generated Texts Impact Term-Based Retrieval Models?
- 发现LLM文本高频词更平滑、低频词更陡峭，术语特异性更高
- 传统检索模型优先匹配查询词分布相似的文档，无源偏见
- 为混合内容检索系统提供可解释性基础，适合信息检索研究者
随着大语言模型（LLMs）生成的内容大量涌入互联网，信息检索（IR）系统面临区分和处理人类与机器生成文本的挑战。近期研究表明，神经检索器可能偏好LLM生成内容，而经典词项检索器如BM25则倾向于人类写作文档。本文研究了LLM生成内容对词项检索模型的影响，这类模型因其高效性和跨领域鲁棒性备受重视。语言学分析显示，LLM生成文本表现出更平滑的高频词与更陡峭的低频词Zipf斜率，更高的术语特异性以及更大的文档级多样性。这些特征源于LLM在训练中优化读者体验所追求的表达多样性和精准性。研究进一步探讨词项检索模型是否存在来源偏见，结论是：这些模型优先选择词项分布与查询最接近的文档，而非固有地偏向某种来源。本工作为理解并应对混合来源内容下词项检索系统的潜在偏差提供了基础。
原文摘要 · Abstract (English)
As more content generated by large language models (LLMs) floods into the Internet, information retrieval (IR) systems now face the challenge of distinguishing and handling a blend of human-authored and machine-generated texts. Recent studies suggest that neural retrievers may exhibit a preferential inclination toward LLM-generated content, while classic term-based retrievers like BM25 tend to favor human-written documents. This paper investigates the influence of LLM-generated content on term-based retrieval models, which are valued for their efficiency and robust generalization across domains. Our linguistic analysis reveals that LLM-generated texts exhibit smoother high-frequency and steeper low-frequency Zipf slopes, higher term specificity, and greater document-level diversity. These traits are aligned with LLMs being trained to optimize reader experience through diverse and precise expressions. Our study further explores whether term-based retrieval models demonstrate source bias, concluding that these models prioritize documents whose term distributions closely correspond to those of the queries, rather than displaying an inherent source bias. This work provides a foundation for understanding and addressing potential biases in term-based IR systems managing mixed-source content.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。