让大模型推理过程可审计，发现隐藏错误和偏见。
Semi-structured LLM Reasoners Can Be Rigorously Audited
- 用类似Python的半结构化语法记录每步推理输入输出。
- 三种自动审计方法均能有效识别推理错误。
- 保持高准确率的同时提升可解释性，适合可信AI研究者。
尽管大语言模型已具备推理能力，但其输出仍存在难以检测的错误与遗漏，可能掩盖模型偏见。为此，我们提出半结构化推理模型（SSRMs），训练其生成具有命名步骤、标注输入输出的半结构化推理轨迹，采用非执行的类Python语法表示。该结构使推理过程可被自动审计。我们评估了三类审计方法：人工编写的领域特定语言（DSL）审计、由大模型生成的结构化审计，以及基于概率模型的典型性审计。结果显示，所有方法均能有效标记潜在推理错误。重要的是，审计能力未影响模型整体性能：在十二个基准测试和两个模型系列上的评估表明，SSRMs在同规模模型中表现优异且具有强泛化能力。
原文摘要 · Abstract (English)
Although Large Language Models (LLMs) have become capable reasoners, the problem of faithfulness persists: their reasoning can contain errors and omissions that are difficult to detect and that may obscure biases in model outputs. To address this issue, we introduce Semi-Structured Reasoning Models (SSRMs), which are trained to produce semi-structured representations of reasoning. SSRMs generate reasoning traces in a non-executable Pythonic syntax that names each reasoning step and marks its inputs and outputs. This structure allows SSRM traces to be automatically audited to identify reasoning flaws. We evaluate three types of audits: hand-crafted structured reasoning audits, written in a domain-specific language (DSL) implemented in Python; LLM-generated structured reasoning audits; and learned typicality audits, which apply probabilistic models over reasoning traces. We show that all of these methods can be used to effectively flag probable reasoning errors. Importantly, the auditability of SSRMs does not appear to compromise overall accuracy: in evaluation on twelve benchmarks and two model families, SSRMs demonstrate strong performance and generalizability relative to other models of comparable size.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。