研究生成式AI如何改变搜索,发现其结果来源与传统搜索差异巨大。
How Generative AI Disrupts Search: An Empirical Study of Google Search, Gemini, and AI Overviews

- 对比谷歌搜索、AI摘要和Gemini,分析生成式搜索的呈现机制。
- 51.5%真实查询生成AI摘要,且多来自谷歌自有内容。
- 网页被屏蔽爬虫后更难被AI摘要收录,稳定性也较差。
生成式AI正越来越多地融入网络搜索,提升用户体验。本文通过构建包含11,500个用户查询的公开基准数据集,研究生成式AI如何改变信息检索与呈现方式。我们比较了谷歌搜索、配套的AI摘要(AIO)以及Gemini Flash 2.5对同一查询的返回结果。发现:51.5%的代表性真实查询会生成AIO,且常出现在有机搜索结果之上;三者检索来源差异显著(平均杰卡德相似度<0.2),传统搜索更倾向权威网站,而生成式引擎则更偏好谷歌自有内容;即使网页内容可访问,若被谷歌AI爬虫屏蔽,则更难被AIO收录;此外,AIO在重复查询中表现不一致,对微小查询改动也较敏感。这些发现对理解生成式搜索对网站可见性、优化策略及用户信息获取的影响具有重要意义,呼吁建立可持续的收益框架,促进内容创作者与生成式搜索平台共赢。
原文摘要 · Abstract (English)
Generative AI is being increasingly integrated into web search for the convenience it provides users. In this work, we aim to understand how generative AI disrupts web search by retrieving and presenting the information and sources differently from traditional search engines. We introduce a public benchmark dataset of 11,500 user queries to support our study and future research of generative search. We compare the search results returned by Google's search engine, the accompanying AI Overview (AIO), and Gemini Flash 2.5 for each query. We have made several key findings. First, we find that for 51.5\% of representative, real-user queries, AIOs are generated, and are displayed above the organic search results. Controversial questions frequently result in an AIO. Second, we show that the retrieved sources are substantially different for each search engine (<0.2 average Jaccard similarity). Traditional Google search is significantly more likely to retrieve information from popular or institutional websites in government or education, while generative search engines are significantly more likely to retrieve Google-owned content. Third, we observe that websites that block Google's AI crawler are significantly less likely to be retrieved by AIOs, despite having access to the content. Finally, AIOs are less consistent when processing two runs of the same query, and are less robust to minor query edits. Our findings have important implications for understanding how generative search impacts website visibility, the effectiveness of generative engine optimization techniques, and the information users receive. We call for revenue frameworks to foster a sustainable and mutually beneficial ecosystem for publishers and generative search providers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。