用廉价探针预判3D CT编码器的训练效果，省下大量计算资源。
When Do Cheap Probes Predict Expensive Training? Probing 3D-CT Encoders for Text Generation
- 设计轻量探针CheapCT，评估编码器表征能力
- 探针与完整微调结果相关性高达ρ=0.90~1.00
- 可快速筛选最优编码器，适合医疗视觉语言模型研发
构建3D CT视觉语言模型需选择合适的图像编码器，传统方法需对每个候选编码器进行完整语言模型微调并比较下游性能，成本极高。本文提出廉价探针CheapCT，测试其是否能预测昂贵训练的结果。在报告生成和新构建的MeasureVQA（基于分割掩码和亨氏单位测量答案）任务上验证，探针在各项能力上均能准确预测微调表现，探针与微调结果的相关性从ρ=0.90到1.00。使用该探针仅需微调单个候选编码器，即可选出接近最优的模型，节省数个数量级的计算开销。代码与MeasureVQA数据集已开源。
原文摘要 · Abstract (English)
Building a 3D CT vision language model begins with a choice of which image encoder to build on. Today that choice is made by fine-tuning every candidate through the full language model and comparing downstream scores, an enormously expensive search. A cheap probe on the encoder's representation promises a way out, but whether it forecasts the expensive outcome has never been tested. We test this with CheapCT on report generation and on MeasureVQA, a new VQA dataset we build. MeasureVQA scores the outcome one capability at a time, its answers measured from segmentation masks and Hounsfield units. Report generation scores the whole report at once and reflects mostly disease. The probe forecasts expensive training across every capability. The rank agreement between probe and fine-tuning stays high throughout, from $ρ=0.90$ to $1.00$. Used to choose an encoder, CheapCT picks one nearly as good as the best while fine-tuning a single candidate, at orders of magnitude less compute. We release the code and MeasureVQA at https://github.com/renjie-liang/CheapCT
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。