用少量参数冻结编码器，大幅提升肺部X光多模态分类效果。
Fixed-Budget Parameter-Efficient Training with Frozen Encoders Improves Multimodal Chest X-Ray Classification
- 冻结视觉编码器，仅训练少量新增参数实现高效微调。
- 在237万参数预算下，模型性能达0.892~0.908 AUROC，是全量微调的1.1倍。
- 适合医疗影像部署，尤其资源受限场景下的快速模型优化。
多模态胸部X光分析常需微调大型视觉语言模型，计算成本高。本文研究参数高效训练（PET）策略，包括冻结编码器、BitFit、LoRA和适配器，在印第安纳大学胸部X光数据集（3,851张图像-报告对，579个测试样本）上进行多标签分类。为避免数据泄露，将报告中的病灶术语移除但保留临床语境。在固定参数预算（237万参数，占总量2.51%）下，所有PET方法均达到0.892~0.908的AUROC，优于使用9430万可训练参数的全量微调（0.770 AUROC），实现40倍参数缩减。在更大规模的CheXpert数据集（224,316张图像，58倍大）上验证可扩展性：所有PET方法均实现>0.69 AUROC，且可训练参数<9%，其中适配器表现最佳（0.7214 AUROC）。预算匹配对比显示，仅视觉模型（0.653 AUROC，106万参数）优于同预算的多模态模型（0.641 AUROC），表明性能提升主要来自参数分配而非跨模态协同。尽管PET方法校准性能下降（ECE: 0.29–0.34，相较简单模型的0.049），但可通过后处理校准解决。结果表明，冻结编码器策略可在显著降低计算开销的同时实现更优判别能力，但临床应用需辅以校准修正。
原文摘要 · Abstract (English)
Multimodal chest X-Ray analysis often fine-tunes large vision-language models, which is computationally costly. We study parameter-efficient training (PET) strategies, including frozen encoders, BitFit, LoRA, and adapters for multi-label classification on the Indiana University Chest X-Ray dataset (3,851 image-report pairs; 579 test samples). To mitigate data leakage, we redact pathology terms from reports used as text inputs while retaining clinical context. Under a fixed parameter budget (2.37M parameters, 2.51% of total), all PET variants achieve AUROC between 0.892 and 0.908, outperforming full fine-tuning (0.770 AUROC), which uses 94.3M trainable parameters, a 40x reduction. External validation on CheXpert (224,316 images, 58x larger) confirms scalability: all PET methods achieve >0.69 AUROC with <9% trainable parameters, with Adapter achieving best performance (0.7214 AUROC). Budget-matched comparisons reveal that vision-only models (0.653 AUROC, 1.06M parameters) outperform budget-matched multimodal models (0.641 AUROC, 1.06M parameters), indicating improvements arise primarily from parameter allocation rather than cross-modal synergy. While PET methods show degraded calibration (ECE: 0.29-0.34) compared to simpler models (ECE: 0.049), this represents a tractable limitation addressable through post-hoc calibration methods. These findings demonstrate that frozen encoder strategies provide superior discrimination at substantially reduced computational cost, though calibration correction is essential for clinical deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。