审计谷歌AI摘要与精选片段，发现孕婴信息存在33%不一致且医疗保障严重缺失。
Auditing Google's AI Overviews and Featured Snippets: A Case Study on Baby Care and Pregnancy
- 通过1508个真实查询系统评估，分析信息一致性与医疗安全
- 33%的搜索结果中AI摘要与精选片段内容矛盾，医疗防护仅11%~7%
- 适合关注AI健康信息质量、公共政策制定者及医疗传播研究者
谷歌搜索越来越多地通过AI摘要（AIO）和精选片段（FS）展示AI生成内容，用户频繁依赖但无法控制其呈现方式。通过对1,508个真实孕婴相关查询进行系统性算法审计，我们评估了这些信息展示在答案一致性、相关性、医疗安全措施、来源类别和情感匹配等方面的质量。结果显示，33%的同一页面中AIO与FS内容存在不一致；尽管相关性评分较高，但两类功能均严重缺乏医疗安全保障，仅11%的AIO和7%的FS包含医疗安全信息。虽然健康类网站是主要来源，但精选片段也常链接至商业来源。该发现对公共卫生信息获取有重要影响，凸显了在高风险领域加强AI信息质量管控的必要性。本研究方法可迁移应用于其他直接影响用户福祉的高风险领域中的AI系统审计。
原文摘要 · Abstract (English)
Google Search increasingly surfaces AI-generated content through features like AI Overviews (AIO) and Featured Snippets (FS), which users frequently rely on despite having no control over their presentation. Through a systematic algorithm audit of 1,508 real baby care and pregnancy-related queries, we evaluate the quality and consistency of these information displays. Our robust evaluation framework assesses multiple quality dimensions, including answer consistency, relevance, presence of medical safeguards, source categories, and sentiment alignment. Our results reveal concerning gaps in information consistency, with information in AIO and FS displayed on the same search result page being inconsistent with each other in 33% of cases. Despite high relevance scores, both features critically lack medical safeguards (present in just 11% of AIO and 7% of FS responses). While health and wellness websites dominate source categories for both, AIO and FS, FS also often link to commercial sources. These findings have important implications for public health information access and demonstrate the need for stronger quality controls in AI-mediated health information. Our methodology provides a transferable framework for auditing AI systems across high-stakes domains where information quality directly impacts user well-being.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。