构建医疗长对话基准，测试大模型对患者长期病史的推理能力
MedLoCoMo: A Long-Context Multi-Session Medical Dialogue Benchmark for Large Language Models
- 基于MIMIC-IV数据构建跨住院会话的临床对话数据集
- 跨会话推理难度显著高于单一会话，即使有长上下文或外部记忆
- 适合研究医疗大模型纵向病史理解与决策能力的研究者
MedLoCoMo 是一个面向患者特定临床推理的医疗长上下文记忆基准，用于评估大语言模型在多住院会话中的表现。现有医疗问答基准多聚焦短上下文知识或单文档定位，未检验大模型是否能有效利用、关联并适时放弃对患者长期病史的推理。我们基于去标识化的 MIMIC-IV 及 MIMIC-IV-Note 数据，构建了以住院为单位的临床数据包，合成真实医生-患者对话，并生成与证据相关的问答项，覆盖单次住院、跨住院及对抗性不可答场景。该基准包含 100 名患者的时间线，平均 1,669.8 轮对话，29.7 次会话，每条对话平均 74,512.2 个词元。实验表明，跨住院推理始终比局部证据使用更困难，即便模型具备长上下文窗口或外部记忆/检索方法。代码与基准数据已开源，便于复现。
原文摘要 · Abstract (English)
MedLoCoMo is a Medical Long-Context Memory benchmark for patient-specific clinical reasoning over multi-admission medical dialogue. Existing medical QA benchmarks largely test short context knowledge or single document grounding, leaving open whether LLMs can use, connect, and abstain over longitudinal patient histories. We build MedLoCoMo from deidentified MIMIC-IV and MIMIC-IV-Note records by constructing admission-level clinical packets, synthesizing grounded doctor-patient conversations, and generating evidence linked QA items over single-admission, cross-admission, and adversarial unanswerable settings. The benchmark contains 100 patient timelines averaging 1,669.8 turns, 29.7 sessions, and 74,512.2 tokens per conversation. Across the evaluated baselines, cross-admission reasoning is consistently harder than localized evidence use, even when models have long context windows or use external memory or retrieval methods. The code and MedLoCoMo benchmark release is available at https://github.com/leozzy13/MedLoCoMo for use and reproducibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。