LLM正在悄然影响维基百科内容，可能扭曲NLP评估结果。
Wikipedia in the Era of LLMs: Evolution and Risks
- 分析维基百科内容与访问数据，结合模拟研究其受LLM影响
- 特定类别内容受LLM影响约1%，导致翻译与RAG性能下降
- 适合关注NLP数据污染风险的研究者和平台维护者
本文提出一个全面的分析与监测框架，研究大语言模型（LLMs）对维基百科的影响。通过分析文章内容与页面浏览量，考察维基百科近期变化并评估LLM的作用。随后，评估LLM对维基百科相关自然语言处理任务（如机器翻译和检索增强生成，RAG）的影响。研究发现，某些类别内容受LLM影响约1%；若基于维基百科的机器翻译基准被污染，模型得分可能虚高，模型间对比结果可能失真；若知识库被LLM污染，RAG的有效性可能降低。尽管当前尚未彻底改变维基百科的语言与知识结构，但实证结果警示需警惕未来潜在风险。实验数据与源代码已公开于：https://github.com/HSM316/LLM_Wikipedia。
原文摘要 · Abstract (English)
In this paper, we present a comprehensive analysis and monitoring framework for the impact of Large Language Models (LLMs) on Wikipedia, examining the evolution of Wikipedia through existing data and using simulations to explore potential risks. We begin by analyzing article content and page views to study the recent changes in Wikipedia and assess the impact of LLMs. Subsequently, we evaluate how LLMs affect various Natural Language Processing (NLP) tasks related to Wikipedia, including machine translation and retrieval-augmented generation (RAG). Our findings and simulation results reveal that Wikipedia articles have been affected by LLMs, with an impact of approximately 1% in certain categories. If the machine translation benchmark based on Wikipedia is influenced by LLMs, the scores of the models may become inflated, and the comparative results among models could shift. Moreover, the effectiveness of RAG might decrease if the knowledge has been contaminated by LLMs. While LLMs have not yet fully changed Wikipedia's language and knowledge structures, we believe that our empirical findings signal the need for careful consideration of potential future risks in NLP research. We release all the experimental dataset and source code at: https://github.com/HSM316/LLM_Wikipedia
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。