揭示序列模型因缺失外部观察者导致的判断偏差,提出可量化纠正机制。
The Need for an External Observer Formalizing the Sufficiency Gap: A Mathematical Extension of Mixture Identifiability and Contextual Grounding in Sequence Models
- 引入隐变量驱动的双模式过程,建模文本与未观测状态的耦合关系。
- 发现即使完美预测文本,仍可能因错误隐状态产生过自信,熵差即为充分性缺口。
- 提出基于可信度γ的外部信号修正机制,适用于高风险场景的模型验证。
我们构建了一个二元混合过程,包含一个确定性的文本模式和一个由未观测隐状态决定的随机模式。即使理想中的无限容量序列预测器能精确恢复仅基于文本的边缘分布,当观测前缀与错误隐状态兼容时,模型仍可能过度自信。由此产生的熵差并非普通优化误差,而是由于对未观测状态的边际化所导致的充分性缺口。随后,我们通过一个辅助二值信号(保真度γ∈[1/2,1])形式化了检索、工具使用和外部校准。该贝叶斯更新产生一个上下文主导阈值:当信号保真度超过文本历史赋予误导性隐状态的后验权重时,纠正信号将逆转先前的后验比。该阈值可缩小但通常无法完全消除充分性缺口;彻底闭合需对相关隐状态实现完美揭示或等效验证机制。分析阐明了温度缩放无法恢复缺失上下文的原因,强调了校准机制必须兼具信息量与可学习性,并指出在高风险领域,自主序列模型需具备结构解耦的外部观察者或验证器。
原文摘要 · Abstract (English)
We construct a binary mixed-regime process with one deterministic textual regime and one random regime governed by an unobserved latent state. Even an ideal infinite-capacity sequence predictor that exactly recovers the text-only marginal law can become overconfident when the observed prefix is compatible with the wrong latent regime. The resulting entropy difference is not an ordinary optimization error; it is a sufficiency gap caused by marginalization over an unobserved state. We then formalize retrieval, tool use, and external grounding through an auxiliary binary signal with fidelity $γ\in [1/2,1]$. The resulting Bayesian update yields a contextual dominance threshold: a corrective signal reverses the posterior odds induced by the textual history exactly when its fidelity exceeds the text-only posterior weight assigned to the misleading regime. This threshold reduces, but does not generally eliminate, the sufficiency gap; complete closure requires perfect revelation of the relevant latent state or an equivalent verification mechanism. The analysis clarifies why temperature scaling cannot restore missing context, why grounding mechanisms must be both informative and learnably usable by the model, and why autonomous sequence models require structurally decoupled observers or verifiers in high-stakes domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。