首个可执行临床指令的肺心音分析语音语言模型
StethoLM: Audio Language Model for Cardiopulmonary Analysis Across Clinical Tasks
- 融合音频编码与医学语言模型，支持多任务指令响应
- 在77,027对指令-回答数据上训练，覆盖7类临床任务
- 适用于基层医疗辅助诊断与医学生培训
听诊心肺音是临床检查中最基础的步骤之一。尽管快速且无创，但准确解读细微音频线索需多年经验。近年来深度学习在自动化心肺音分析方面取得进展，但多数仅限于简单分类，缺乏临床可解释性与决策支持能力。本文提出StethoLM，首个专用于心肺听诊的语音-语言模型，能够执行跨全谱听诊分析的指令驱动临床任务。该模型结合音频编码与医学语言模型主干，并在包含77,027对指令-响应对的StethoBench基准上训练，这些数据源自16,125个标注的心肺录音，涵盖二分类、检测、报告、推理、鉴别诊断、对比和定位分析共七类临床任务。通过监督微调与直接偏好优化的多阶段训练，StethoLM在分布外数据上表现出显著提升的性能与鲁棒性。本工作为听诊领域指令跟随式AI系统奠定了基础。
原文摘要 · Abstract (English)
Listening to heart and lung sounds - auscultation - is one of the first and most fundamental steps in a clinical examination. Despite being fast and non-invasive, it demands years of experience to interpret subtle audio cues. Recent deep learning methods have made progress in automating cardiopulmonary sound analysis, yet most are restricted to simple classification and offer little clinical interpretability or decision support. We present StethoLM, the first audio-language model specialized for cardiopulmonary auscultation, capable of performing instruction-driven clinical tasks across the full spectrum of auscultation analysis. StethoLM integrates audio encoding with a medical language model backbone and is trained on StethoBench, a comprehensive benchmark comprising 77,027 instruction-response pairs synthesized from 16,125 labeled cardiopulmonary recordings spanning seven clinical task categories: binary classification, detection, reporting, reasoning, differential diagnosis, comparison, and location-based analysis. Through multi-stage training that combines supervised fine-tuning and direct preference optimization, StethoLM achieves substantial gains in performance and robustness on out-of-distribution data. Our work establishes a foundation for instruction-following AI systems in clinical auscultation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。