用翻译模型解决大模型输出难验证问题，提升可检性同时不损失准确率。
Mitigating Legibility Tax with Decoupled Prover-Verifier Games
- 分离正确性与可验证性，训练翻译模型转换解题结果为易验证形式
- 在保持原模型准确率前提下，实现可验证性显著提升
- 适合需要高可信输出的场景，如自动化评测与安全推理
随着大语言模型能力增强，其输出被能力较弱系统验证变得愈发关键。当前基于证明者-验证者博弈的方法虽能提升输出可检性，但相比仅优化正确性的基线模型会出现准确率下降，即所谓的可读性税。本文提出解耦策略：先训练一个固定求解器以最大化正确性，再训练一个“翻译”模型将求解器输出转化为易验证形式，同时保留原答案。为此，我们构建了去耦证明者-验证者博弈（DPVG），其均衡对应于忠实且可验证的翻译模型。
原文摘要 · Abstract (English)
As large language models become increasingly capable, it is critical that their outputs can be easily checked by less capable systems. Prover-verifier games can be used to improve checkability of model outputs, but display a degradation in accuracy compared to a baseline trained only to maximize correctness -- a phenonemon named legibility tax. We propose a solution by decoupling the correctness from the checkability condition and instead training a "translator" model that turns a fixed solver model's solution into a checkable form. This allows us to first train the solver to maximize correctness, and then train the translator to translate the solver into a checkable form while retaining the solver's answer. To accommodate this new objective of translation, we formulate a decoupled prover-verifier game (DPVG) where the equilibria correspond to faithful and checkable translators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。