用有限语义证书验证大模型上下文确定性,揭示其推理边界与涌现阈值。
Finite Certificates for In-Context Determinacy and a Threshold Theory of Emergence in Language Models
- 提出有限语义证书替代标签,构建可验证的上下文确定性框架。
- 证明最小强制子上下文提取在二值输出下为NP完全问题。
- 发现阈值跃迁反映语义转变而非评分函数突变,适合理论研究者。
本文建立模型论框架,以有限语义证书替代基准标签,验证上下文条件下的语言模型行为。首要问题是有限确定性:上下文中的示例何时在不修改参数的情况下强制决定查询答案?在有限域线性任务族中,证明了精确的行空间判据,计算残差假设数量,推导出完整与查询局部识别曲线,并表明提取最小强制子上下文即使对二值输出也是NP完全的。第二个问题是阈值涌现:看似基准跳跃是否反映语义转变而非评分映射的不连续?证明反幻象定理,区分阈值度量与语义置信,并给出隐式承诺在阈值以上显现的速率敏感交叉界。共同语义对象是可定义事件上的置信泛函,其为布尔概率测度,等价于相关类型空间上的凯斯勒测度,测度为1的公式构成真滤子,且其石空间表示在定义扩张下不变。由此产生的微积分提供有限上下文证书、成对分离击中集、查询教学维数、提示保全准则与尺度极限见证。精确算术辅助脚本复现有限域与阈值计算,并生成图示数据。
原文摘要 · Abstract (English)
This paper develops a model-theoretic framework for verifying context-conditioned language-model behavior by replacing benchmark labels with finite semantic certificates. The first problem is finite determinacy: when do examples in a context force the answer to a query without changing model parameters? In finite-field linear task families, we prove an exact row-space criterion, compute the residual hypothesis count, derive full and query-local identification curves, and show that extracting a smallest forcing subcontext is NP-complete even for binary outputs. The second problem is threshold emergence: when does an apparent benchmark jump reflect a semantic transition rather than a discontinuity of the scoring map? We prove an anti-mirage theorem separating thresholded metrics from semantic confidence and give a rate-sensitive crossing bound for latent commitments becoming visible above threshold. The common semantic object is a confidence functional on definable events. We show that it is a Boolean probability measure, equivalently a Keisler measure on the relevant type space, whose measure-one formulas form a proper filter and whose Stone-space representation is invariant under definitional expansion. The resulting calculus provides finite context certificates, pair-separator hitting sets, query teaching dimension, prompt-preservation criteria, and scale-limit witnesses. Exact-arithmetic ancillary scripts reproduce the finite-field and threshold calculations and generate the data used by the figures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。