让小模型学会医生式推理,用轻量方法提升医疗视觉问答能力
LiteMedCoT-VL: Parameter-Efficient Adaptation for Medical Visual Question Answering

- 用大模型思维链指导20亿参数小模型,通过LoRA微调实现知识迁移
- 在PMC-VQA上达64.9%准确率,比基线高11个百分点,仅用2B参数超越4B模型
- 不依赖图像描述,真实模拟临床读片场景,适合部署于便携医疗设备
大型与紧凑型视觉语言模型(VLM)之间的推理能力差距,限制了医疗AI在便携临床设备上的部署。2–40亿参数的紧凑型VLM可在资源受限硬件上运行,但缺乏多步推理能力,难以支持可解释的临床决策。现有知识蒸馏方法仅传递答案,未传递推理过程。医疗视觉问答(VQA)是该问题的理想测试平台,因其要求模型通过结构化推理链整合视觉证据与临床知识。我们提出LiteMedCoT-VL,一个将2350亿参数教师模型的思维链推理迁移至20亿参数学生模型的流程,基于LoRA微调并使用带解释的训练数据。所有推理默认不使用图像字幕,模拟医生直接解读医学图像而无放射报告的临床场景。在PMC-VQA基准上,LiteMedCoT-VL达到64.9%准确率,相较零样本Qwen3-VL-4B基线(53.9%)提升11.0个百分点,优于所有已发表基线。结果表明,经推理蒸馏的20亿参数模型可媲美甚至超过参数量翻倍的模型。视觉定位分析显示,模型依赖图像内容,而非利用文本先验。代码已公开于https://anonymous.4open.science/r/LiteMedCoT-VL。
原文摘要 · Abstract (English)
The reasoning gap between large and compact vision-language models (VLMs) limits the deployment of medical AI on portable clinical devices. Compact VLMs of 2--4B parameters can run on resource-constrained hardware but lack the multi-step reasoning capacity needed for interpretable clinical decision support. Existing knowledge distillation methods transfer answers without the reasoning process behind them. Medical visual question answering (VQA) serves as a testbed for this problem, as it requires models to integrate visual evidence with clinical knowledge through structured reasoning chains. We introduce LiteMedCoT-VL, a pipeline that transfers chain-of-thought reasoning from a 235B teacher model to 2B student models through LoRA-based fine-tuning on explanation-enriched training data. All inference is conducted without image captions by default, simulating the clinical scenario in which a physician interprets a medical image directly without an accompanying radiology report. On the PMC-VQA benchmark, LiteMedCoT-VL achieves 64.9% accuracy, exceeding the zero-shot Qwen3-VL-4B baseline of 53.9% by 11.0 percentage points and outperforming all published baselines. This result indicates that a 2B model with reasoning distillation can match or exceed models with twice the parameters. Visual grounding analysis shows that the model relies on image content rather than exploiting textual priors. Our code is publicly available at https://anonymous.4open.science/r/LiteMedCoT-VL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。