自动合成精简可执行验证器，提升大模型输出的可信度
AutoPyVerifier: Learning Compact Executable Verifiers for Large Language Model Outputs
- 用LLM生成候选验证函数，再通过有向无环图搜索优化
- 在多个基准上使验证准确率最高提升55.0 F1点
- 生成的验证器可作为外部工具增强下游任务表现
验证正成为基于强化学习训练和推理时控制大语言模型（LLMs）的核心。然而当前验证器存在根本权衡：基于LLM的验证器表达能力强但难控且易错，而确定性可执行验证器可靠且可解释但能力有限。我们提出：给定目标任务（如正确性）的模型输出及标签数据集，能否自动生成一组最小化的Python验证器，使其联合满足度逼近目标？我们提出AutoPyVerifier框架，利用LLM生成候选验证函数，并通过有向无环图（DAG）搜索进行优化。该方法系统探索确定性可执行验证器空间，选出最紧凑的验证器集合，使其联合满足度最优匹配目标。在多个先进大模型的数学推理、编码、函数调用和指令遵循基准测试中，相较于初始的LLM生成验证器集，目标预测F1值最高提升55.0点。分析显示，不同任务和模型的最佳验证目标各异，且基于DAG的搜索促使学习到的验证器更侧重结构与语义层面检查。进一步实验表明，将发现的验证器集作为外部工具提供给大模型，可使下游任务准确率最高提升17.0点。代码已开源。
原文摘要 · Abstract (English)
Verification is becoming central to both reinforcement-learning-based training and inference-time control of large language models (LLMs). Yet current verifiers face a fundamental trade-off: LLM-based verifiers are expressive but hard to control and prone to error, while deterministic executable verifiers are reliable and interpretable but often limited in capability. We study the following question: given a development set of LLM outputs and labels for a target objective, such as correctness, can we automatically induce a minimal set of Python verifiers whose joint satisfaction closely matches that objective? We propose AutoPyVerifier, a framework that uses an LLM to synthesize candidate verifier functions and then refines them through search over a directed acyclic graph (DAG). By navigating the DAG, AutoPyVerifier systematically explores the space of deterministic executable verifiers and selects a compact verifier set whose joint satisfaction best approximates the target objective. Across mathematical reasoning, coding, function calling, and instruction-following benchmarks for several state-of-the-art LLMs, AutoPyVerifier improves target-objective prediction by up to 55.0 F1 points over the initial LLM-generated verifier sets. Additional analyses show that the most useful verification targets vary by benchmark and model, and that the DAG-based search shifts the learned verifier sets toward more structural and semantically grounded checks. We further show that exposing the discovered verifier set to an LLM as an external tool improves downstream accuracy by up to 17.0 points. We release our code
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。