提出双锚点框架，让医学视觉语言模型更抗提示干扰。
BiomedAP: A Vision-Informed Dual-Anchor Framework with Gated Cross-Modal Fusion for Robust Medical Vision-Language Adaptation
- 用双锚点约束和门控跨模态融合，动态调节噪声信息。
- 在11个基准上实现比基线更高的少样本准确率与鲁棒性。
- 适合临床少样本诊断场景，尤其对不规范提示有强适应力。
生物医学视觉-语言模型在少样本医疗诊断中表现优异，但面临关键瓶颈：对提示变化敏感。现有适配框架通常将视觉与文本提示作为独立流优化，依赖理想“黄金提示”。在临床现实中，描述常含噪声且异质，这种模态隔离导致跨模态对齐不稳定。为此，我们提出BiomedAP，一种视觉引导的双锚点框架，结合门控跨模态融合。BiomedAP通过两种机制实现协同对齐：(1) 门控跨模态融合，在层间实现模态交互，动态抑制无关文本线索；(2) 双锚点约束，将可学习提示正则化至来自专家模板（高锚点）和少样本视觉原型（低锚点）的稳定语义中心。11个基准的广泛实验表明，BiomedAP持续优于基线，在提示扰动下显著提升鲁棒性与少样本准确率。代码已开源：https://github.com/tongdiedie/BiomedAP。
原文摘要 · Abstract (English)
Biomedical Vision--Language Models (VLMs) have shown remarkable promise in few-shot medical diagnosis but face a critical bottleneck: \textit{fragility to prompt variations}.Existing adaptation frameworks typically optimize visual and textual prompts as independent streams, relying on ideal ``Golden Prompts''. In clinical reality, where descriptions are often noisy and heterogeneous, this modality isolation leads to unstable cross-modal alignment. To address this, we propose BiomedAP, a vision-informed dual-anchor framework with gated cross-modal fusion.BiomedAP enforces synergistic alignment through two mechanisms: (1) Gated Cross-Modal Fusion, which enables layer-wise interaction between modalities, acting as a dynamic noise regulator to suppress irrelevant textual cues; and (2) a Dual-Anchor Constraint that regularizes learnable prompts toward stable semantic centroids derived from both expert templates (High Anchors) and few-shot visual prototypes (Low Anchors). Extensive experiments across 11 benchmarks demonstrate that BiomedAP consistently surpasses baselines, achieving competitive few-shot accuracy and markedly enhanced robustness under prompt perturbations. Our code is available at: https://github.com/tongdiedie/BiomedAP. Keywords: Vision-Language Models; Prompt Learning; Parameter-Efficient Fine-Tuning; Few-shot Learning
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。