arXiv:2512.19602cs.CV2025-12被引 1

让医学影像与表格数据联合学习在缺失时依然稳定,适合真实医疗场景。

No Data? No Problem: Robust Vision-Tabular Learning with Missing Values

  • 用缺失数据做增强训练,提升模型对不完整表格的鲁棒性。
  • 在英国生物银行心脏影像上,全缺失到全可用场景下性能更稳定。
  • 适用于医疗、广告等多领域,尤其适合表格数据常缺失的场景。

大规模医学生物银行提供影像数据和丰富的表格信息(如临床指标或人口统计)。然而,现实数据中仅部分属性可用,与理想情况不符。为此,我们提出RoVTL框架,可应对0%至100%任意缺失程度的表格数据。该框架包含两个阶段:对比预训练阶段引入表格属性缺失作为数据增强以提升鲁棒性;下游任务微调阶段采用新型Tabular More vs. Fewer损失函数,根据可用表格数据量对性能进行排序。结合门控交叉注意力融合模块,该方法在所有表格数据完整度下均保持一致性能。我们在英国生物银行的心脏MRI数据集上评估,相比现有方法显著提升对缺失表格数据的鲁棒性。此外,RoVTL在外部心脏影像数据集上实现多模态疾病分类,并拓展至自然图像领域,在汽车广告数据集上也表现出稳健性能。模型权重与代码已公开于https://github.com/marteczkah/RoVTL。

原文摘要 · Abstract (English)

Large-scale medical biobanks provide imaging data complemented by extensive tabular information, such as clinical measurements or demographics. However, this abundance of tabular attributes does not reflect real-world datasets, where only a subset of attributes may be available. This discrepancy calls for methods that remain robust to missing values at inference. To address this challenge, we propose RoVTL (Robust Vision-Tabular Learning), a framework designed to handle any level of tabular data availability, from 0% to 100%. RoVTL comprises two key stages: contrastive pretraining, where we introduce tabular attribute missingness as data augmentation to promote robustness, and downstream task tuning, where tabular missingness is complemented by a novel Tabular More vs. Fewer loss that ranks performance based on the amount of available tabular data. Combined with gated-cross attention fusion module, our tuning approach enables consistent performance across all tabular data completeness scenarios. We evaluate RoVTL on cardiac MRI scans from the UK Biobank, demonstrating superior robustness to missing tabular data compared to prior methods. Furthermore, RoVTL successfully generalizes to an external cardiac MRI dataset for multimodal disease classification, and extends to the natural images domain, achieving robust performance on a car advertisements dataset. The model weights and code are available at https://github.com/marteczkah/RoVTL.

多模态学习缺失数据医疗影像鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。