用知识图谱可视化调试医疗大模型诊断逻辑,发现三类常见错误。
VeriLLMed: Interactive Visual Debugging of Medical Large Language Models with Knowledge Graphs

- 将模型推理路径转为可比形式,结合知识图谱构建标准参考路径。
- 识别出关系错、分支错、遗漏错三类高频诊断错误。
- 适合医疗AI开发者和临床研究者用于提升模型可靠性。
大型语言模型在医疗诊断中展现出潜力,但因临床决策风险高且推理可靠性不足,实际部署仍面临挑战。当前调试医学LLM困难重重:开发者缺乏医学背景难以理解错误的临床意义;模型故障涉及多种输入类型、任务和推理步骤,难以确定优先排查项;且现有方法以单个实例为中心,难发现共性错误模式。为此,我们提出VeriLLMed,一个融合外部生物医学知识的可视化分析系统,用于审计与调试医疗LLM的诊断推理。该系统将模型输出转化为可比较的推理路径,基于知识图谱构建参考路径,并识别出三类典型诊断错误:关系错误、分支错误和遗漏错误。案例研究与专家评估表明,VeriLLMed能帮助开发者发现临床不合理的推理过程,并生成可操作的改进洞察,从而提升医疗大模型的可靠性。
原文摘要 · Abstract (English)
Large language models (LLMs) show promise in medical diagnosis, but real-world deployment remains challenging due to high-stakes clinical decisions and imperfect reasoning reliability. As a result, careful inspection of model behavior is essential for assessing whether diagnostic reasoning is reliable and clinically grounded. However, debugging medical LLMs remains difficult. First, developers often lack sufficient medical domain expertise to interpret model errors in clinically meaningful terms. Second, models can fail across a large and diverse set of instances involving different input types, tasks, and reasoning steps, making it challenging for developers to prioritize which errors deserve focused inspection. Third, developers struggle to identify recurring error patterns across cases, as existing debugging practices are largely instance-centric and rely on manual inspection of isolated failures. To address these challenges, we present VeriLLMed, a visual analytics system that integrates external biomedical knowledge to audit and debug medical LLM diagnostic reasoning. VeriLLMed transforms model outputs into comparable reasoning paths, constructs knowledge graph-grounded reference paths, and identifies three recurring classes of diagnosis errors: relation errors, branch errors, and missing errors. Case studies and expert evaluation demonstrate that VeriLLMed helps developers identify clinically implausible reasoning and generate actionable insights that can inform the improvement of medical LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。