让AI自动生成可复用的临床推理代码,减少人工维护负担。
CodeClinic: Evaluating Automation of Coding Skills for Clinical Reasoning Agents

- 构建MIMIC-IV基准,评估LLM合成临床技能能力
- 自动生成的代码库使推理更一致,查询耗时降低40%
- 适合医疗AI研究者和临床决策系统开发者
基于大语言模型的临床推理代理旨在自动化重症监护室监测与电子病历中的患者状态追踪。现有系统通常依赖人工编写的临床工具或技能(如脓毒症检测、器官功能衰竭评估),但维护这些工具库需大量专家投入,而零样本查询或代码生成常导致推理链低效且不可靠,尤其在机构特异性临床政策下表现更差。我们提出CodeClinic,一个基于MIMIC-IV的基准,用于评估LLM代理是否能自主合成与组合可复用的临床技能,而非依赖固定工具箱。该基准包含两项互补任务:纵向ICU监测和组合式信息检索。纵向场景模拟每四小时进行一次结构化决策,覆盖25个临床发现和8类临床问题;组合式场景涵盖63,000个实例,涉及259项任务,分九个领域,并按组合依赖深度分层,以评估复杂多步推理能力。我们还提出离线自动形式化流程,通过迭代式LLM优化,将自然语言临床指南转化为可复用且经过验证的Python技能库。相比零样本代码生成,新生成的代码库显著提升一致性,同时每查询令牌消耗减少高达40%。
原文摘要 · Abstract (English)
Clinical reasoning agents based on large language models (LLMs) aim to automate tasks such as intensive care unit (ICU) monitoring and patient state tracking from electronic health records (EHRs). Existing systems typically rely on manually curated clinical tools or skills for concepts such as sepsis detection and organ failure assessment. However, maintaining these tool libraries requires substantial expert effort, while zero-shot querying or code generation often produces inefficient and unreliable reasoning chains, especially under institution-specific clinical policies. We introduce CodeClinic, a benchmark built on MIMIC-IV for evaluating whether LLM agents can synthesize and compose reusable clinical skills instead of relying on fixed toolboxes. The benchmark contains two complementary tasks: longitudinal ICU surveillance and compositional information seeking. The longitudinal setting simulates monitoring patient trajectories with structured decisions every four hours across 25 findings and eight clinical families, while the compositional setting spans 63k instances across 259 tasks in nine domains and is stratified by compositional dependency depth to evaluate increasingly complex multi-step reasoning. We further propose an offline autoformalization pipeline that converts natural-language clinical guidelines into reusable and verified Python skill libraries through iterative LLM refinement. Compared with zero-shot code generation, the resulting libraries improve consistency while reducing per-query token usage by up to 40%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。