arXiv:2508.01450cs.CL2025-08中稿 · CVPR被引 3

用少量数据高效训练医学视觉语言模型,提升临床推理能力。

Towards Efficient Medical Reasoning with Minimal Fine-Tuning Data

  • 基于难度与梯度影响双指标筛选高质量样本
  • 仅用1%数据达到全量数据性能,10%优于传统方法
  • 适合医疗AI研究者和低资源场景下的模型优化

监督微调在将视觉语言模型适配至医学推理等专业领域中起关键作用。现有方法常依赖未经筛选的文本数据,导致计算成本高且在复杂临床场景下表现不佳。尽管已有方法根据样本难度(知识与推理复杂度)选择数据,却忽略了其优化价值——梯度影响。我们发现,仅依赖梯度影响会偏向易优化但缺乏深层推理的样本,而仅依据难度则选出噪声多或过于复杂的案例。为此,提出难度-影响象限(DIQ)策略,优先选择高难度、高梯度影响的样本,平衡复杂推理与显著参数更新。实验证明,使用仅1%的DIQ选中数据即可达到全数据集性能,10%数据持续超越基线。人类与LLM评价显示,该策略生成的推理更贴近专家实践,在鉴别诊断、安全检查与证据引用方面表现更优。代码已开源。

原文摘要 · Abstract (English)

Supervised Fine-Tuning (SFT) of the language backbone plays a pivotal role in adapting Vision-Language Models (VLMs) to specialized domains such as medical reasoning. However, existing SFT practices often rely on unfiltered textual datasets that contain redundant and low-quality samples, leading to substantial computational costs and suboptimal performance in complex clinical scenarios. Although existing methods attempt to alleviate this problem by selecting data based on sample difficulty, defined by knowledge and reasoning complexity, they overlook each sample's optimization utility reflected in its gradient. Interestingly, we find that gradient-based influence alone favors easy-to-optimize samples that cause large parameter shifts but lack deep reasoning chains, while difficulty alone selects noisy or overly complex textual cases that fail to guide stable optimization. Based on this observation, we propose a data selection strategy, Difficulty-Influence Quadrant (DIQ), which prioritizes samples in the "high-difficulty-high-influence" quadrant to balance complex clinical reasoning with substantial gradient influence. This enables efficient medical reasoning for VLMs with minimal fine-tuning data. Furthermore, Human and LLM-as-a-judge evaluations show that DIQ-selected subsets demonstrate higher data quality and generate clinical reasoning that is more aligned with expert practices in differential diagnosis, safety check, and evidence citation, as DIQ emphasizes samples that foster expert-like reasoning patterns. Extensive experiments on medical reasoning benchmarks demonstrate that DIQ enables VLM backbones fine-tuned on only 1% of selected data to match full-dataset performance, while using 10% consistently outperforms baseline methods, highlighting the superiority of principled data selection over brute-force scaling. The code is available at https://github.com/mihara-bot/DIQ.

医学推理数据筛选小样本学习视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。