检测网页是否由大模型主导生成，识别率高达100%。
Preprint: Poster: Did I Just Browse A Website Written by LLMs?
- 通过分析网站中类散文页面的文本特征，实现整站分类。
- 在120个真实网站上达到100%准确率，野外检测发现大量大模型生成网站。
- 适合关注网络内容真实性、搜索引擎生态的研究者与平台方。
越来越多的网页内容由大语言模型（LLMs）自动生成，人类参与极少，这类内容被称为“大模型主导型”内容。由于大模型存在抄袭和幻觉问题，此类内容可能不可靠且不道德。然而，网站通常不披露内容来源，人类读者也难以辨别。因此亟需可靠的检测工具。但现有大模型检测器在网页内容上表现不佳，因网页具有低正样本率、复杂标记结构和多样文体，而非干净的散文式基准数据。我们提出一种高效可靠的整站检测流水线：不直接对每页文本进行分类，而是基于多个类散文页面的检测结果，综合判断整个网站。通过收集两个总计120个网站的真实标签数据集进行训练与评估，检测准确率达100%。在实际场景中，我们在1万条搜索结果和1万条Common Crawl归档数据中识别出大量大模型主导网站，发现其数量持续增长且排名靠前，引发对用户及整个网络生态影响的担忧。
原文摘要 · Abstract (English)
Increasingly, web content is automatically generated by large language models (LLMs) with little human input. We call this "LLM-dominant" content. Since LLMs plagiarize and hallucinate, LLM-dominant content can be unreliable and unethical. Yet, websites rarely disclose such content, and human readers struggle to distinguish it. Thus, we must develop reliable detectors for LLM-dominant content. However, state-of-the-art LLM detectors are inaccurate on web content, because web content has low positive rates, complex markup, and diverse genres, instead of clean, prose-like benchmark data SoTA detectors are optimized for. We propose a highly reliable, scalable pipeline that classifies entire websites. Instead of naively classifying text extracted from each page, we classify each site based on an LLM text detector's outputs of multiple prose-like pages to boost accuracies. We train and evaluate our detector by collecting 2 distinct ground truth datasets totaling 120 sites, and obtain 100% accuracies testing across them. In the wild, we detect a sizable portion of sites as LLM-dominant among 10k sites in search engine results and 10k in Common Crawl archives. We find LLM-dominant sites are growing in prevalence and rank highly in search results, raising questions about their impact on end users and the overall Web ecosystem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。