arXiv:2601.14227cs.SD2026-01被引 1

用Transformer提升呼吸音分析准确率,实现更客观的肺部疾病筛查。

Transformer Architectures for Respiratory Sound Analysis and Multimodal Diagnosis

  • 采用预训练Audio Spectrogram Transformer模型,结合自注意力机制分析呼吸音。
  • 在哮喘检测中达到97%准确率、97% F1分数和0.98 AUC,显著优于传统方法。
  • 首次将临床信息融合进诊断流程,适合开发智能辅助诊疗系统。

呼吸音分析是筛查哮喘等肺部疾病的重要手段,但传统听诊主观性强、依赖经验。此前研究基于DenseNet201构建了CNN基准模型,表现良好。本文(i)将Audio Spectrogram Transformer(AST)应用于呼吸音分析,并(ii)评估一个整合声谱图与结构化患者信息的多模态视觉-语言模型(VLM)。AST使用公开预训练权重,在包含每类诊断数百条录音的医疗数据集上微调。VLM采用轻量级Moondream型模型,输入声谱图图像及性别、年龄、采集部位等文本提示,输出JSON格式诊断结果。实验显示,AST在哮喘检测中达到约97%准确率、97% F1分数和0.98 ROC AUC,显著优于内部CNN基线及常见外部基准;VLM达到86%-87%准确率,与CNN相当,且能有效融合临床上下文信息。结果验证了自注意力机制在声学筛查中的有效性,凸显多模态架构在构建综合诊断工具方面的潜力。

原文摘要 · Abstract (English)

Respiratory sound analysis is a crucial tool for screening asthma and other pulmonary pathologies, yet traditional auscultation remains subjective and experience-dependent. Our prior research established a CNN baseline using DenseNet201, which demonstrated high sensitivity in classifying respiratory sounds. In this work, we (i) adapt the Audio Spectrogram Transformer (AST) for respiratory sound analysis and (ii) evaluate a multimodal Vision-Language Model (VLM) that integrates spectrograms with structured patient metadata. AST is initialized from publicly available weights and fine-tuned on a medical dataset containing hundreds of recordings per diagnosis. The VLM experiment uses a compact Moondream-type model that processes spectrogram images alongside a structured text prompt (sex, age, recording site) to output a JSON-formatted diagnosis. Results indicate that AST achieves approximately 97% accuracy with an F1-score around 97% and ROC AUC of 0.98 for asthma detection, significantly outperforming both the internal CNN baseline and typical external benchmarks. The VLM reaches 86-87% accuracy, performing comparably to the CNN baseline while demonstrating the capability to integrate clinical context into the inference process. These results confirm the effectiveness of self-attention for acoustic screening and highlight the potential of multimodal architectures for holistic diagnostic tools.

呼吸音分析Transformer多模态诊断

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。