LLM工具在压缩会话时误把超时输出当真实结果，导致错误持续传播。
Compaction as Epistemic Failure: How Agentic LLM Tools Fabricate Confirmed Results from Killed Processes
- 将超时命令的临时输出误作持久结果存入摘要
- 错误结果跨会话、跨版本传播，未被重新验证
- 适合关注AI自动化可靠性与验证机制的研究者
具有自主能力的LLM编程工具会将长会话历史压缩为摘要，供后续会话继承作为事实依据。本文揭示了Claude Code存在一种故障模式：当命令因超时退出（退出码143）时，其部分标准输出会被错误地记录在压缩摘要中，被视为已确认的结果。这种错误会跨会话和模型版本持续传播，且不经过重新验证。根本原因是将终端中的临时观察误认为持久存储的信息。该发现扩展了先前关于LLM自评估失败的研究，表明自主型工具在报告自身运行结果时同样存在可靠性缺陷。这一问题直接影响依赖会话连续性的数据处理、科学计算和多步自动化流程。
原文摘要 · Abstract (English)
Agentic LLM coding tools compress long session histories into compaction summaries that subsequent sessions inherit as ground truth. This paper documents a failure mode in Claude Code where partial standard output from timed-out commands (exit code 143) is recorded in compaction summaries as confirmed results, propagating false positives across sessions and model versions without re-verification. The underlying mechanism is a conflation of observation and persistence, where information that appeared in the terminal is treated as equivalent to information written to durable storage. This finding extends the analysis of LLM self-evaluation failures reported in prior work on non-determinism in LLM-as-judge grading by showing that agentic tools exhibit analogous reliability deficits when reporting on their own operational outcomes. The failure has direct implications for any workflow that relies on agentic session continuity for data processing, scientific computation, or multi-step automation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。