让大模型决策可解释、可验证,提升临床试验匹配的可信度。
Accountable AI with Grounded, Faithful, Consistent, Actionable Rationales: A Case Study in Clinical Trial Matching with VERDICT

- 将决策规则转为逻辑公式,用数学求解器确保一致执行
- 在两个数据集上准确率超越现有模型,且政策执行零偏差
- 生成医生认可的推理,明确指出改变结果的关键条件
问责性要求决策可审查、可辩护、可质疑。大模型虽输出流畅,但常缺乏依据、不完整或与决策过程不符。实现问责需具备可验证的推理(如何得出结论)、假设说明(哪些是推测而非事实)、政策一致性(相同情况相同处理)及关键条件(什么会改变结果)。本文提出自一致性测试作为自动问责检验:改变关键条件应导致决策变化。以高风险的临床试验匹配任务为例,尽管现有大模型匹配效果尚可,但决策策略不一致,推理也不忠实于自身判断。为此提出VERDICT,将任务、约束和策略转化为满足模理论(SMT)问题,通过SMT与MaxSMT求解器推导决策——确保策略严格一致,且决策天生可问责。在基于SIGIR 2016和TREC 2021的数据集上,VERDICT在仅使用大模型和神经符号基线中达到最高决策准确率,政策执行完美一致,且生成的推理被临床医生偏好,基于显式假设与关键条件,反事实自一致性显著提升。
原文摘要 · Abstract (English)
Accountability means a decision can be examined, justified, and contested. LLMs make this hard: fluent output may be ungrounded, incomplete, or unfaithful to the decision process. Achieving accountability requires verified rationales (how was the decision reached), assumptions (what was assumed rather than known), policy consistency (the same treatment for the same facts), and pivotal conditions (what would change the outcome). We introduce self-faithfulness as an automatic test of accountability: changing the pivotal conditions should change the decision. We examine accountable AI through clinical trial matching, a high-stakes task central to evidence-based medicine. Although LLM-based matchers match patients to trials reasonably accurately, they apply decision policies inconsistently and produce rationales that are unfaithful to their own decisions. We introduce VERDICT, an LLM-based agent that translates a decision task, its constraints, and its policy into Satisfiability Modulo Theories (SMT), then derives the decision with SMT and MaxSMT solvers -- so policies are applied consistently and decisions are accountable by construction. Across a SIGIR 2016-derived dataset and TREC 2021, VERDICT achieves the strongest decision accuracy among LLM-only and neurosymbolic baselines, applies policies with perfect consistency, and produces clinician-preferred rationales grounded in explicit assumptions and pivotal conditions, with improved counterfactual self-faithfulness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。