给大模型验证机制分五个等级，搞清谁说了算、能保证什么。
Grading the Graders: Verification Autonomy Levels (L0-L5) for LLM Reasoning
- 按验证依据来源和结论保证力，划分从L0到L5的五级验证自主性标准
- 发现多数验证器只能保证正确性（L2），无法证明没有遗漏答案
- 适用于评估大模型推理验证能力的研究者与开发者
大型语言模型（LLM）越来越多地搭配验证器（步骤检查器、自一致性过滤器、基于工具的事实核查器、形式化证明助手）来检测错误。然而，现有文献中“等级”一词含义混乱，涵盖验证粒度、概念抽象、风险层级、系统层级及真值来源等五种不同维度。本文提出验证自主性等级（VAL），统一以验证规范来源与判决保证力为单一轴线，划分从L0（模型自我声明；无确定锚点）到L2（客观真值；仅保证正确性），再到L3/L4（可判定系统，具备单属性或领域完整性），而L5在无限制情况下不可达。核心是完整性盲区：基于替换与采样的验证器可确认候选成立，但无法证明未遗漏其他候选。我们进一步指出文献未明言的二分法：完整性仅对形式可规约属性可达，而经验性开放世界验证（如事实核查、诊断）最多只能达到有锚定的正确性（L2）。我们在四个领域（符号数学、行为监控、医疗诊断、代码生成，后者为逆向验证，预测先于证据）中实证验证此差距，并在最强的形式验证基线中观察到作者承认验证聚焦于每步正确性。我们证明粒度、概念层次、风险与系统栈等级与VAL正交，澄清了17篇调研论文中的系统性混淆。代码与完整评估作为补充材料公开。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly paired with verifiers (step checkers, self-consistency filters, tool-based fact checkers, formal proof assistants) that claim to detect the model's errors. Yet the verification literature uses the word "level" to mean at least five different things: verification granularity, concept abstraction, risk tier, system-stack layer, and the epistemic source of the ground truth. We propose Verification Autonomy Levels (VAL), a meta-standard that classifies any verification scheme along a single axis: where does the verification spec come from, and what does the verdict guarantee? VAL ranges from L0 (LLM self-declaration; no deterministic anchor) through L2 (objective ground truth; correctness only) to L3/L4 (decidable systems with single-property or domain-level completeness), with L5 impossible in the unrestricted case. Central to VAL is the completeness blind spot: substitution- and sampling-based verifiers can confirm that proposed candidates hold, but cannot prove that no candidate was missed. We further identify a dichotomy the literature has not stated: completeness is reachable only for formally specifiable properties, whereas empirical open-world verification (fact-checking, diagnosis) caps at anchored correctness (L2). We document this gap empirically across four domains (symbolic mathematics, behavior monitoring, medical diagnosis, and code generation, the last a reverse validation with predictions stated before evidence) and in the strongest formal-verification baseline in our survey, whose authors note the verifier focuses on the correctness of each step. We show the levels of granularity, concept hierarchy, risk, and system stack are orthogonal to VAL, resolving a systematic conflation across 17 surveyed papers. Code and full assessment are released as supplementary material.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。