梳理多服务商大模型服务中的故障模式，揭示静默故障的严重性。
FailureAtlas: A Taxonomy of Failure Modes in Multi-Provider LLM Serving Infrastructure
- 构建双轴分类框架，按故障来源与可检测性划分
- 发现静默故障占比高，返回200状态却导致数据损坏
- 适合运维、系统架构师参考，提升大模型服务可靠性
多服务商大模型网关作为请求路由、负载均衡和限流的关键生产基础设施，其特有的故障模式尚未被系统记录。本文提出 a{}，一个基于故障起源层（网络/传输、流式/协议、状态/会话、模型行为、治理/成本）与可检测性（响亮/静默）的双轴分类体系。通过公开漏洞报告与实地压力测试，我们验证了五个故障案例，每个均附有根因分析；其中三个提供独立复现脚本。核心发现：最严重的故障为静默型——返回HTTP 200，通过所有标准健康检查，但导致应用状态语义级污染。在 b{} 评估中首次发现两个典型静默故障：并发竞争条件引发对话历史丢失，流式索引冲突导致工具调用载荷错误。
原文摘要 · Abstract (English)
Multi-provider LLM gateways reverse proxies that route, load-balance, and rate-limit requests across foundation-model APIs have become critical production infrastructure. Yet the failure modes specific to this architectural layer remain undocumented, scattered across issue trackers and post-mortems with no unifying framework. We introduce \fa{}, a two-axis taxonomy that classifies failures by their \emph{origin layer} (Network/Transport, Streaming/Protocol, State/Session, Model~Behavior, Governance/Cost) and their \emph{detectability} (Loud vs.\ Silent). We populate this taxonomy with five verified catalog entries sourced from public bug reports and first-hand stress testing, each accompanied by a mechanistic root-cause analysis. Three entries include standalone reproduction scripts. Our principal finding is that the most operationally severe failures are \emph{silent}: they return HTTP~200, pass every standard health check, and corrupt application state in ways that require semantic-level observability to detect. Two such silent failures a concurrency race condition causing history loss and a streaming index collision corrupting tool-call payloads were discovered first-hand during \cb{} evaluation campaigns.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。