用声音+病历联合建模，提升呼吸系统疾病诊断准确率
RespiraMFM: A Multimodal Foundation Model with Contrastive Audio-Language Alignment for Respiratory Disease Identification
- 融合呼吸音与病史症状，通过对比学习对齐多模态信息
- 在监督和零样本场景下分别提升9.15%和20.98%的诊断性能
- 适合医疗AI研发者、临床辅助诊断系统开发者使用
呼吸系统疾病仍是全球主要死亡原因之一，及时准确的诊断对改善患者预后、减轻医疗负担至关重要。以往基于音频的单模态方法普遍存在泛化能力差、诊断精度不足的问题。本文提出RespiraMFM，一种整合呼吸音与患者病史、症状的多模态基础模型，采用对比对齐策略实现音频与文本临床信息的跨模态表征学习。我们在七个真实世界数据集上评估了RespiraMFM在五类主要呼吸系统疾病上的表现，涵盖监督微调与零样本设置。结果显示，在监督任务中AUROC提升9.15%，在零样本任务中提升20.98%，显著优于现有基线。这些成果证明该框架在推动早期诊断与优化临床决策方面的潜力。
原文摘要 · Abstract (English)
Respiratory diseases remain a leading cause of global mortality, where timely and accurate diagnosis is critical to improving patient outcomes and reducing healthcare burdens. While prior work has explored audio-based models for respiratory disease detection, such unimodal approaches often suffer from limited generalizability and diagnostic precision. In this paper, we propose RespiraMFM, a Multimodal Foundation Model that integrates respiratory sounds with patient medical history and symptoms to enhance diagnostic accuracy and disease detection capabilities. We introduce an effective contrastive alignment strategy for audio-text multimodal integration, allowing the model to learn better cross-modal representations between respiratory sounds and corresponding textual clinical information. We evaluate RespiraMFM across five major respiratory diseases using seven real-world datasets in both supervised fine-tuning and zero-shot settings, achieving a 9.15% improvement in AUROC on supervised tasks and a 20.98% gain on zero-shot tasks over existing baselines. These findings underscore the potential of our framework to advance early diagnosis and improve clinical decision-making in respiratory disease management.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。