分析生产环境大模型代理系统中的沉默故障，揭示其五大类型及致命风险。
When Errors Become Narratives: A Longitudinal Taxonomy of Silent Failures in a Production LLM Agent Runtime
- 构建五类故障分类体系，聚焦错误信号未被人类察觉的隐蔽问题。
- 70%沉默故障靠用户观察发现，测试与审计仅能阻断87%的回归问题。
- 提出防御框架，强调故障应可追溯、可归因且不具迷惑性。
大型语言模型代理系统在生产环境中长期运行：调度任务、调用工具、维护记忆并推送结果给用户。本研究对一个自2026年3月起持续运行的个人助理代理系统进行纵向分析，该系统包含约40个定时任务、8个LLM提供商、工具治理代理和知识库记忆平面，受4,286个单元测试和827个治理检查保护。在八周内记录22起事故并完成完整根因复盘，其中一种“错误信号无法以可操作形式传递给人类”的元模式出现至少28次。我们提出五类机制导向的分类体系：(A) 环境与平台异常，(B) 设计假设偏差，(C) 错误吞没与稀释，(D) 链式幻觉与虚构，(E) 操作遗漏与取证盲区。类别D为大模型系统独有且最危险：系统不仅不报告错误，还让大模型将其转化为流畅可信的叙述呈现给用户。我们称之为fail-plausible——失败的可观测性被放大为欺骗性误导。三项发现：约70%的沉默故障由用户观察捕获，非测试或审计；对15起事故的回溯审计显示，0%可事前预防，但87%能阻断回归；事故延迟时间（13小时至60天）与故障机制相关，而非代码复杂度——最长的故障存在于组件间缝隙，无测试覆盖。本文描述了相应防御框架，并提炼出使代理系统故障“响亮、可归因、平淡”的设计原则。所有复盘文档与数据公开可用。
原文摘要 · Abstract (English)
LLM agent systems increasingly run as long-lived autonomous runtimes: scheduling jobs, calling tools, maintaining memory, and pushing results to humans. We present a longitudinal study of silent failures in one such system: a personal-assistant agent runtime in continuous production since March 2026, with roughly 40 scheduled jobs, 8 LLM providers, a tool-governance proxy, and a knowledge-base memory plane, defended by 4,286 unit tests and 827 governance checks. Over eight weeks we documented 22 incidents with full root-cause postmortems, in which one meta-pattern -- a failure whose error signal never reaches a human in actionable form -- manifested at least 28 times. We derive a five-class, mechanism-oriented taxonomy: (A) environment and platform quirks, (B) design-assumption mismatches, (C) error swallowing and dilution, (D) chained hallucination and fabrication, (E) operational omission and forensic blind spots. Class D is unique to LLM systems and the most dangerous: the system does not merely fail to report an error -- the LLM transforms it into fluent, plausible narrative delivered to the user. We term this fail-plausible: gray failure's differential observability escalated -- the observer is not just blind, it is convincingly lied to by the failure itself. Three findings: about 70% of silent failures were caught by human user-view observation, not tests or audits; a retrospective audit of 15 incidents found 0% ex-ante prevention but 87% regression blocking -- audits are regression engines, not prediction engines; incident latency (13 hours to 60 days) tracks failure mechanism, not code complexity -- the longest-lived failures lived in the seams between components, where no test runs. We describe the resulting defense framework and distill design principles for agent systems whose failures are loud, attributable, and boring. All postmortems and artifacts are public.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。