提出对话系统持续记忆的理论基石,揭示信息压缩与流失的内在规律。
The Root Theorem of Context Engineering
- 从上下文窗口有限性出发,推导出最大化信号/令牌比的核心原则。
- 证明仅追加内容的系统终将超限,而压缩重写是持久记忆的唯一途径。
- 适用于长期对话系统设计,尤其适合构建稳定连续的记忆架构。
任何需在多轮会话中维持大语言模型对话的系统都面临两个不可逃避的约束:上下文窗口有限,且信息质量随累积量下降。我们将其形式化为公理,推导出上下文工程的根定理——在有损、容量受限的信道中最大化信号/令牌比。由此不附加假设地得出五个结论:(1) 质量函数F(P)随注入令牌量单调下降,与窗口大小无关;(2) 信号与令牌数可独立优化;(3) 触发机制由保真度阈值决定,而非容量限制;(4) 必然出现稳态持续性——积累、压缩、重写、舍弃,是实现无限理解的唯一架构;(5) 压缩机制作用于自身所压缩的信道,需外部验证门。我们证明,仅追加的系统在有限时间内必然超出有效窗口,检索增强生成解决搜索问题但无法保障连续性。该定理的约束结构与生物记忆架构通过独立推导达成一致。通过一个运行超过60轮会话的持久架构验证了工程可行性,实现了稳定内存占用。根定理将上下文工程确立为具有严格信息论基础的学科,区别于提示工程的范畴与方法。香农解决了点对点传输,上下文工程解决了连续性。
原文摘要 · Abstract (English)
Every system that maintains a large language model conversation beyond a single session faces two inescapable constraints: the context window is finite, and information quality degrades with accumulated volume. We formalize these constraints as axioms and derive a single governing principle -- the Root Theorem of Context Engineering: \emph{maximize signal-to-token ratio within bounded, lossy channels.} From this principle, we derive five consequences without additional assumptions: (1)~a quality function $F(P)$ that degrades monotonically with injected token volume, independent of window size; (2)~the independence of signal and token count as optimization variables; (3)~a necessary gate mechanism triggered by fidelity thresholds, not capacity limits; (4)~the inevitability of homeostatic persistence -- accumulate, compress, rewrite, shed -- as the only architecture that sustains understanding indefinitely; and (5)~the self-referential property that the compression mechanism operates inside the channel it compresses, requiring an external verification gate. We show that append-only systems necessarily exceed their effective window in finite time, that retrieval-augmented generation solves search but not continuity, and that the theorem's constraint structure converges with biological memory architecture through independent derivation from shared principles. Engineering proof is provided through a 60+-session persistent architecture demonstrating stable memory footprint under continuous operation -- the divergence prediction made concrete. The Root Theorem establishes context engineering as an information-theoretic discipline with formal foundations, distinct from prompt engineering in both scope and method. Shannon solved point-to-point transmission. Context engineering solves continuity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。