arXiv:2608.28599cs.AI2026-08

用反事实奖励提升医疗诊断效率,减少检查次数与成本。

CDPR: Counterfactual Advantage-based Credit Assignment for Cost-Aware Sequential Medical Diagnosis

论文配图:CDPR: Counterfactual Advantage-based Credit Assignment for Cost-Aware Sequential Medical Diagnosis
图 1 · 摘自论文原文
  • 基于不确定性识别犹豫状态,用短轨迹对比选择优势
  • 在MIMIC-IV等三数据集上降低检查数量与成本,提升准确率
  • 无需专家标签或额外评论器,适合临床决策系统研发者

临床诊断是逐步进行、考虑成本的过程:医生逐项安排检查,根据结果更新诊断,最终得出结论。现有医学语言模型多将诊断视为单次分类任务,忽视检测价值与成本的权衡。本文将诊断建模为成本感知的序列决策过程,采用强化学习训练策略。核心挑战在于信用分配:仅在长轨迹末端获得可靠信号,导致冗余检查与高效检查得分相同。为此提出CDPR(反事实诊断过程奖励),无需专家标签或学习到的评论器。CDPR首先通过动作分布不确定性识别策略犹豫状态,再通过短轨迹滚动估计所选动作相对于政策自身考虑的其他动作的优势,滚动目标平衡正确性、检查次数、成本与不可行请求。批内轨迹缓存复用以降低计算开销。将CDPR集成至GRPO,在一个域内(MIMIC-IV)和两个域外(ClinicalBench及某私立医院数据集)基准上测试,结果表明其在提升诊断准确率的同时,显著减少检查数量与成本。

原文摘要 · Abstract (English)

Clinical diagnosis is a step-by-step, cost-aware process: a physician orders examinations one at a time, observes the results, and updates the diagnosis before reaching a final conclusion. Most medical language models instead treat diagnosis as a one-pass classification task and ignore the trade-off between a test's value and its cost. We model diagnosis as a cost-aware sequential decision process and train the policy with reinforcement learning. The main difficulty is credit assignment: the only reliable signal comes once at the end of a long trajectory, so it scores a wasteful workup the same as an efficient one. We propose CDPR (Counterfactual Diagnostic Process Reward), which needs no expert labels and no learned critic. CDPR first finds the states where the policy hesitates, using the uncertainty of its action distribution, and then scores the chosen action by its advantage over the alternatives the policy itself would consider, estimated with short rollouts under a utility that balances correctness against test count, cost, and infeasible requests. A rollout cache reuses within-batch trajectories to keep the cost low. We integrate CDPR into GRPO and test it on one in-domain (MIMIC-IV) and two out-of-domain (ClinicalBench and a private hospital dataset) benchmarks. CDPR improves diagnostic accuracy while clearly reducing the number and cost of examinations.

医疗诊断强化学习成本优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。