用真实医疗反馈训练临床智能体，发现并解决学习瓶颈。
World Feedback for Clinical Agents: Diagnosing RL in FHIR Environments
- 构建MAB-v3数据集，量化医疗任务可学性
- 纯强化学习仅达18.2%成功率，低于规则微调的34.1%
- 提出决策/格式/查表三类分类法指导模型改进
临床协议执行任务——如检查检验值、设定阈值、生成结构化FHIR订单——天然适合通过世界反馈进行强化学习：一旦临床专家将判断逻辑编码为验证器，即可对无限次试运行进行无标注评分。但强化学习需可靠反馈通道与足够基础能力。我们审计MedAgentBench v1/v2，发现41.7%的静默完成上限使不作为成为强化学习主导策略，并构建了新版本MedAgentBench-v3（508个任务，8.9%成功率上限）。训练Qwen3-8B暴露两大结构性障碍：能力天花板（20类任务中有10类基础性能为0%，无梯度）与格式知识壁垒（20类中3类需精确临床代码，探索无法发现）。纯强化学习仅达18.2%的pass@1，而基于规则的SFT达34.1%，15.9个百分点差距完全由上述障碍导致。提出决策/格式/查表分类法，预测强化学习可学性并建议：用SFT注入代码，用RL学习条件逻辑。
原文摘要 · Abstract (English)
Clinical protocol-execution tasks -- checking a lab value, applying a threshold, placing a correctly structured FHIR order -- are natural candidates for RL from world feedback: once clinical SMEs encode decision logic into a verifier, that verifier grades unlimited rollouts without per-episode annotation. But applying RL requires a sound feedback channel and sufficient base capability. We audit MedAgentBench v1/v2, find a 41.7\% silent-finish ceiling that makes inaction the RL dominant strategy, and construct \textbf{MedAgentBench-v3 (MAB-v3)} (508 tasks, 8.9\% ceiling). Training Qwen3-8B exposes two structural barriers: a \emph{capability ceiling} (10/20 task types have 0\% base performance, zero gradient) and a \emph{format-knowledge barrier} (3/20 types require exact clinical codes undiscoverable by exploration). Pure RL reaches 18.2\% pass@1 vs.\ 34.1\% for rule-based SFT; the 15.9~pp gap is attributable entirely to these barriers. A decision/format-knowledge/lookup taxonomy predicts RL learnability and prescribes the fix: SFT to inject codes, RL to learn conditionals.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。