arXiv:2508.16839cs.AI2025-08被引 3

用一个视觉语言模型分阶段智能路由和专病部署,提升临床模型效率

One VLM, Two Roles: Stage-Wise Routing and Specialty-Level Deployment for Clinical Workflows

  • 分三阶段路由:模态→异常类型→模型卡,支持提前终止与校准筛选
  • 路由准确率提升9~11个百分点,校准误差降低,跨专科部署性能接近专用模型
  • 适合临床研发团队、医疗AI系统集成者,提升可维护性与透明度

临床机器学习流程常因碎片化而低效:分诊、任务选择与模型部署依赖多个特定任务网络。这些流程通常不符合数据科学实践,导致效率低下且运维成本高,还缺乏基于影像/表格数据的模型自动识别及标准化输出。本文提出一种框架,仅使用一个视觉语言模型(VLM)在两种互补模块中发挥作用。第一(方案1):该VLM作为智能模型匹配器,通过三阶段流程(模态→主要异常→模型卡ID)将图像路由至合适专家模型。通过阶段式提示实现“无”或“其他”早期终止,并采用校准后的前两名答案选择器与阶段阈值,使路由准确率在训练集和保留集上分别提升9和11个百分点,同时改善保留集校准性(更低的期望校准误差,ECE)。第二(方案2):对同一VLM在专病数据集上微调,实现每专科一个模型覆盖多个下游任务,简化部署且保持性能。在胃肠科、血液科、眼科、病理学与放射科中,单模型部署表现达到或接近专用基线。两项方案独立可运行,组合后为从分诊到部署提供了一条实用、模块化的路径。

原文摘要 · Abstract (English)

Clinical ML workflows are often fragmented and inefficient: triage, task selection, and model deployment are handled by a patchwork of task-specific networks. These pipelines are rarely aligned with data-science practice, reducing efficiency and increasing operational cost. They also lack data-driven model identification (from imaging/tabular inputs) and standardized delivery of model outputs. We present a framework that employs a single vision-language model (VLM) in two complementary, modular roles. First (Solution 1): the VLM acts as an aware model-card matcher that routes an incoming image to the appropriate specialist model via a three-stage workflow (modality -> primary abnormality -> model-card ID). Reliability is improved by (i) stage-wise prompts enabling early termination via "None"/"Other" and (ii) a calibrated top-2 answer selector with a stage-wise cutoff. This raises routing accuracy by +9 and +11 percentage points on the training and held-out splits, respectively, compared with a baseline router, and improves held-out calibration (lower Expected Calibration Error, ECE). Second (Solution 2): we fine-tune the same VLM on specialty-specific datasets so that one model per specialty covers multiple downstream tasks, simplifying deployment while maintaining performance. Across gastroenterology, hematology, ophthalmology, pathology, and radiology, this single-model deployment matches or approaches specialized baselines. Together, these solutions reduce data-science effort through more accurate selection, simplify monitoring and maintenance by consolidating task-specific models, and increase transparency via per-stage justifications and calibrated thresholds. Each solution stands alone, and in combination they offer a practical, modular path from triage to deployment.

临床AI多模态模型路由部署优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。