arXiv:2603.19843cs.CYcs.CL2026-03中稿 · EMNLP被引 2

AI助检索提升效率但易致过度依赖,错误信息下仍自信满满。

Overreliance on AI in Information-seeking from Video Content

  • 用人类实验对比纯视频、有AI辅助、有误导AI三种场景
  • 无视频时AI助准率提27%-35%,但遇假信息准确率暴跌32%
  • 用户对答案信心不降反升,暴露严重安全风险

多媒体内容的普及正在重塑在线信息环境,尤其在社交媒体中。与此同时,生成式AI正迅速改变搜索方式,大语言模型(LLMs)被广泛用作用户与多媒体内容之间的中介,以检索和总结信息。然而,LLM的不准确性及其潜在漏洞对多媒体信息获取任务的影响仍不明确。我们研究了生成式AI对视频信息检索中的准确性、效率和信心的影响。通过约900名参与者完成8,000多个基于视频的信息检索任务,比较三种条件:(1)仅访问视频,(2)视频+基于LLM的AI辅助,(3)视频+提供虚假答案的欺骗性AI助手。结果显示,当参与者观看了相关视频片段时,AI辅助使准确率提高3%-7%;未观看时则提升27%-35%。短视频效率提升10%,长视频提升25%。但参与者倾向于过度依赖AI输出,在面对欺骗性AI时准确率下降高达32%。令人担忧的是,自评信心在所有条件下均保持稳定。研究揭示了AI中介视频信息检索中的根本性安全风险。

原文摘要 · Abstract (English)

The ubiquity of multimedia content is reshaping online information spaces, particularly in social media environments. At the same time, search is being rapidly transformed by generative AI, with large language models (LLMs) routinely deployed as intermediaries between users and multimedia content to retrieve and summarize information. Despite their growing influence, the impact of LLM inaccuracies and potential vulnerabilities on multimedia information-seeking tasks remains largely unexplored. We investigate how generative AI affects accuracy, efficiency, and confidence in information retrieval from videos. We conduct an experiment with around 900 participants on 8,000+ video-based information-seeking tasks, comparing behavior across three conditions: (1) access to videos only, (2) access to videos with LLM-based AI assistance, and (3) access to videos with a deceiving AI assistant designed to provide false answers. We find that AI assistance increases accuracy by 3-7% when participants viewed the relevant video segment, and by 27-35% when they did not. Efficiency increases by 10% for short videos and 25% for longer ones. However, participants tend to over-rely on AI outputs, resulting in accuracy drops of up to 32% when interacting with the deceiving AI. Alarmingly, self-reported confidence in answers remains stable across all three conditions. Our findings expose fundamental safety risks in AI-mediated video information retrieval.

AI可信度信息检索用户行为大模型风险

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。