揭示多智能体大模型流水线中幻觉随阶段递增扩散的机制
The Hallucination Snowball: Modeling Error Propagation as State Transitions in Multi-Agent LLM Pipelines
- 将幻觉传播建模为四状态马尔可夫过程，揭示其不可逆演化规律
- 346次注入幻觉中23.7%在最终输出中完全未被检测到，检测率从72%降至50.9%
- 验证节点位置比有无验证更重要，首阶段衔接处应优先部署校验
序列化多智能体大模型流水线在各阶段间缺乏验证，导致结构性缺陷并产生可量化且严重后果。我们发现，阶段1引入的幻觉不仅持续存在，还会经历转化：原始数值事实演变为推导计算，再转为叙事文本，最终成为编辑认可的结论。每次转换均导致可检测性近乎不可逆地下降。我们将此现象形式化为‘幻觉雪球效应’，构建四状态一阶马尔可夫过程（原始事实→推导→叙事→不可见），实测边界逃逸概率分别为24.6%、48.3%和89.3%。在包含4个智能体的金融分析流水线（FinanceBench）上，针对346次自动注入的幻觉，gpt-4o的检测率从阶段1的72.0%降至阶段4的50.9%，其中23.7%的幻觉完全未被察觉。即使最强模型Qwen3.5-397B-A17B（阶段1检测率87.0%）也面临结构上限，预测阶段4检测率仅约60–65%。关键的是，使用相同RAG验证工具在边界部署检查，可使幻觉存活率从58.4%降至16.2%（与末段检查相比，Cohen's h = -0.911, p < 0.000001），而仅末段检查仅带来2.3个百分点提升。验证的位置比是否验证更为关键。我们的模型可预测n阶段线性流水线的幻觉存活率，并建议最优资源分配：优先在S1→S2阶段投入验证，此时仍有75.4%的幻觉可捕获，而非在S3→S4阶段（已有89.3%逃逸）。
原文摘要 · Abstract (English)
Sequential multi-agent LLM pipelines chain specialized agents without verification at handoffs, creating a structural flaw with measurable and severe consequences. We show that hallucinations injected at Stage 1 do not merely persist; they transform: raw numerical facts become derived computations, then narrative prose, then editorially approved conclusions. At each transformation, detectability degrades near-irreversibly. We formalize this as the hallucination snowball effect, a first-order Markov process over four states (Raw Fact $\to$ Derived $\to$ Narrative $\to$ Invisible) with empirically measured per-boundary escape probabilities of 24.6%, 48.3%, and 89.3%. Across 346 automatically injected hallucinations in a 4-agent financial analysis pipeline on FinanceBench, gpt-4o detection drops from 72.0% at Stage 1 to 50.9% at Stage 4, and 23.7% of hallucinations survive completely undetected in the final output. Even the strongest model tested (Qwen3.5-397B-A17B, 87.0% at Stage 1) faces a structural ceiling; projected Stage 4 detection is only ${\sim}$60--65%. Critically, boundary gates using identical RAG verification tools reduce hallucination survival from 58.4% to 16.2% versus end-of-pipeline checking (Cohen's $h = -0.911$, $p < 0.000001$), while end-checking alone achieves merely 2.3 pp improvement over no verification. When you verify matters more than whether you verify. Our model predicts survival for $n$-agent linear pipelines and prescribes optimal verification resource allocation: invest at $S_1{\to}S_2$ first, where 75.4% of hallucinations are still catchable, not at $S_3{\to}S_4$ where 89.3% have already escaped.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。