让大模型在测试时动态优化判断，提升罕见病影像定位准确率
Dynamic Decision Learning: Test-Time Evolution for Abnormality Grounding in Rare Diseases
- 用指令优化和视觉扰动融合，让冻结模型在测试时自我修正
- 在罕见病数据上将定位准确率提升105%，优于微调和基线方法
- 生成可信度评分，适合医疗诊断中对可靠性要求高的场景
罕见病的临床异常定位常因数据稀缺而受阻，监督微调不可行，单次推理极不稳定。本文提出动态决策学习（DDL），使冻结的大规模视觉-语言模型（LVLMs）能在语言和视觉空间中通过优化指令与整合扰动下的预测结果，持续改进决策。该过程提升了定位质量，并生成基于共识的可靠性评分以量化模型置信度。在脑部影像基准测试中，涵盖281种病理类型的罕见病数据集，模型参数范围从3B到72B，DDL在罕见病案例上将mAP@75提升最高达105%，超越适配基线与监督微调。此外，即使在严重分布偏移和任务难度增加情况下，其可靠性评分与定位精度仍保持更强校准性。代码已公开：https://lijunrio.github.io/DDL/
原文摘要 · Abstract (English)
Clinical abnormality grounding for rare diseases is often hindered by data scarcity, making supervised fine-tuning impractical and single-pass inference highly unstable. We propose Dynamic Decision Learning (DDL), a framework that enables frozen large vision-language models (LVLMs) to refine their decisions across both language and visual spaces by optimizing instructions and consolidating predictions under visual perturbations. This process improves localization quality and produces a consensus-based reliability score that quantifies model confidence. Results on brain imaging benchmarks, including a rare-disease dataset with 281 pathology types across models ranging from 3B to 72B parameters, show that DDL improves mAP@75 by up to 105% on rare-disease cases and outperforms adaptation baselines and supervised fine-tuning. Furthermore, DDL demonstrates stronger calibration between reliability scores and localization accuracy under severe distribution shifts and increasing task difficulty. Code is available at: https://lijunrio.github.io/DDL/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。