用动态超图实现多跳临床推理，提升医学多模态诊断准确率
HyperWalker: Dynamic Hypergraph-Based Deep Diagnosis for Multi-Hop Clinical Modeling across EHR and X-Ray in Medical VLMs
- 构建动态超图iBrochure，融合电子病历与影像的高阶关联
- 通过强化学习代理在超图中搜索最优诊断路径，准确率超越当前最佳
- 适合需要多轮推理和长程依赖建模的医疗AI研究者
自动化临床诊断仍是医疗AI的核心挑战，需整合多模态数据并跨复杂病例上下文推理。尽管现有医学视觉语言模型（VLMs）在医学报告生成（MRG）和视觉问答（VQA）上取得进展，但多数方法采用样本孤立推理范式，独立处理每个病例，无法访问纵向电子健康记录（EHR）或结构相关的患者案例。这种范式仅依赖图像信息，忽略外部互补医学证据。为此，我们提出超图驱动的深度诊断框架HyperWalker，通过动态超图与测试时训练重构临床推理。首先，构建名为iBrochure的动态超图，刻画EHR数据的结构异质性及多模态临床信息间的隐含高阶关联；在此超图中，强化学习代理Walker导航并识别最优诊断路径。为确保测试样本覆盖多样临床特征，引入滞留机制——一种多跳正交检索策略，迭代选取反映不同临床属性的邻近病例。在MIMIC上的MRG和EHRXQA上的医疗VQA实验表明，HyperWalker达到当前最优性能。代码已开源。
原文摘要 · Abstract (English)
Automated clinical diagnosis remains a core challenge in medical AI, which usually requires models to integrate multi-modal data and reason across complex, case-specific contexts. Although recent methods have advanced medical report generation (MRG) and visual question answering (VQA) with medical vision-language models (VLMs), these methods, however, predominantly operate under a sample-isolated inference paradigm, as such processing cases independently without access to longitudinal electronic health records (EHRs) or structurally related patient examples. This paradigm limits reasoning to image-derived information alone, which ignores external complementary medical evidence for potentially more accurate diagnosis. To overcome this limitation, we propose \textbf{HyperWalker}, a \textit{Deep Diagnosis} framework that reformulates clinical reasoning via dynamic hypergraphs and test-time training. First, we construct a dynamic hypergraph, termed \textbf{iBrochure}, to model the structural heterogeneity of EHR data and implicit high-order associations among multimodal clinical information. Within this hypergraph, a reinforcement learning agent, \textbf{Walker}, navigates to and identifies optimal diagnostic paths. To ensure comprehensive coverage of diverse clinical characteristics in test samples, we incorporate a \textit{linger mechanism}, a multi-hop orthogonal retrieval strategy that iteratively selects clinically complementary neighborhood cases reflecting distinct clinical attributes. Experiments on MRG with MIMIC and medical VQA on EHRXQA demonstrate that HyperWalker achieves state-of-the-art performance. Code is available at: https://github.com/Bean-Young/HyperWalker
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。