大模型可靠性问题可局部解决,关键在发现并覆盖常见错误模式。
The Architecture of Errors: From Universal Impossibility to Patch-Local LLM Reliability
- 将全局不可靠性问题转化为局部错误模式的发现与干预覆盖问题。
- 在特定任务场景中,只需少量干预即可显著提升可靠性,且成本随长度增长缓慢。
- 适合关注实际部署中模型稳定性与错误修复的研究者和工程师。
通用大模型的可靠性并非有限词典可解:在所有可能的任务、工具、数据模式、知识源及评估期望下,新的可区分失败模式会无限涌现,因此任何有限干预字典都无法保证对所有模式的残余误差有界。但实际部署系统运行于操作边界明确的特定场景(如法律审查、医疗RAG、代码修复、客服代理、合同提取)中,其任务、模式、工具和评估标准具有重复性。实证表明,失败模式稀疏、重复且集中于少数常见类型,可靠性问题因而变为局部模式发现与干预覆盖问题,而非指数级的上下文长度难题。本文通过两个命题与一个推论形式化这一转变:命题1为最坏情况下的否定结果——有限干预字典无法覆盖无界域中的所有可区分失败模式;推论1指出,模式发现的对数上界无法容纳线性增长的尾部模式,除非观测到指数级的硬失败事件;命题2为正向的局部结果——在对活跃模式的对数暴露量与头部覆盖的前提下,足够的每硬决策干预预算随序列长度呈多对数增长,一旦场景模式库饱和,该预算即趋于领域常数。该框架并未消除长上下文困难,而是将问题定位至可干预的轴线上,而非使难问题变易。
原文摘要 · Abstract (English)
Universal LLM reliability is not a finite-library problem: across all possible tasks, tools, schemas, knowledge sources, and evaluator expectations, new intervention-distinguishable failure modes can appear without bound, so no finite intervention dictionary can guarantee bounded residual error for every such mode. But deployed systems do not operate over the whole universe. They operate inside operationally bounded patches (legal review, medical RAG, code repair, customer-support agents, contract extraction) with recurring tasks, schemas, tools, and evaluator expectations. Within such patches, empirical evidence suggests failures are sparse, repetitive, and concentrated in a small recurring catalogue, so reliability becomes a local catalogue-discovery and intervention-coverage problem rather than an exponential token-length problem. We formalize this transition with two propositions and one corollary. Proposition 1 is the worst-case-mode-wise negative result: no finite intervention dictionary covers every distinguishable failure mode of an unbounded domain. Corollary 1 is the inverse-discovery implication: the logarithmic upper bound on mode discovery cannot accommodate linearly more distinct tail modes without exponentially more observed hard-failure events. Proposition 2 is the positive patch-local result: under log active-mode exposure and head-heavy coverage, a sufficient per-hard-decision intervention budget grows polylogarithmically in sequence length and becomes domain-constant once the patch catalogue saturates. The framework relocates rather than dissolves long-context difficulty: where the number of hard decisions itself grows with task length, reliability remains hard; the contribution is to identify the on-axis intervention rather than to make those regimes easy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。