分析10万次人机交互，发现AI失败大多无声无息，提出八类失效模式。
Invisible failures in human-AI interactions
- 通过分析真实对话数据，提炼出8种用户未察觉的隐形失败模式。
- 79%的AI失败无明显迹象，即使大模型升级后仍多数不可见。
- 该分类可帮助开发者、研究者识别系统性缺陷，适合产品监控场景。
AI系统比可见失败更常以无声方式失效。在对10万次来自WildChat数据集的人机交互分析中，我们发现79%的AI失败是隐形的：出现问题但用户未表现出异常。这些隐形失败可归纳为8种原型，揭示了系统在满足用户需求时的具体失灵点。此外，原型之间存在系统性共现模式，反映出更高层级的失败类型。为检验这些原型在更先进模型中的适用性，我们构建了一个反事实数据集，用当前前沿大模型（frontier LMs）替换原2024年模型的回复。结果表明，失败率显著下降，但绝大多数失败仍保持不可见状态，且失败原型分布稳定。最后，我们展示如何利用这些原型识别不同使用场景下的系统性与可变性局限。总体而言，本研究提出的隐形失败分类体系可成为产品开发者、科研人员和政策制定者进行可靠故障监测的关键工具。代码与数据已公开于https://github.com/bigspinai/bigspin-invisible-failure-archetypes。
原文摘要 · Abstract (English)
AI systems fail silently far more often than they fail visibly. In an analysis of 100K human-AI interactions from the WildChat dataset, we find that 79% of AI failures are invisible: something went wrong but the user gave no overt indication that there was a problem. These invisible failures cluster into eight archetypes that help us characterize where and how AI systems are failing to meet users' needs. In addition, the archetypes show systematic co-occurrence patterns indicating higher-level failure types. To address the question of whether these archetypes will remain relevant as AI systems become more capable, we also created and annotated a counterfactual dataset in which WildChat's 2024-era responses are replaced by those from three present-day frontier LMs. This analysis indicates that failure rates have dropped substantially, but that the vast majority of failures remain invisible in our sense, and the distribution of failure archetypes seems stable. Finally, we illustrate how the archetypes help us to identify systematic and variable AI limitations across different usage domains. Overall, we argue that our invisible failure taxonomy can be a key component in reliable failure monitoring for product developers, scientists, and policy makers. Our code and data are available at https://github.com/bigspinai/bigspin-invisible-failure-archetypes
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。