用智能代理生成呼吸音并诊断疾病,解决数据少和信息丢失问题。
Resp-Agent: An Agent-Based System for Multimodal Respiratory Sound Generation and Disease Diagnosis
- 用主动学习的智能体动态调度合成任务,闭环优化诊断能力。
- 在22.9万条数据上训练,提升罕见病诊断准确率,应对数据不均衡。
- 适合医疗AI研究者,尤其关注声音诊断与多模态生成的人。
基于深度学习的呼吸音听诊目前面临两大挑战:(i) 将信号转换为频谱图会丢失瞬时声学事件和临床背景信息;(ii) 数据量有限,且类别严重不平衡。为此,我们提出Resp-Agent,一个由新型主动对抗式课程智能体(Thinker-A$^2$CA)驱动的自主多模态系统。不同于静态流程,Thinker-A$^2$CA作为中央控制器,主动识别诊断弱点并以闭环方式调度针对性合成。为弥合表示差距,我们引入一种模态编织诊断器,通过全局注意力与稀疏音频锚点将临床文本与音频标记融合,捕捉长程临床上下文和毫秒级瞬变。为缓解数据不足,设计了一种流匹配生成器,通过模态注入将仅文本的大型语言模型(LLM)适配,解耦病理内容与声学风格,以合成难诊断样本。本工作还构建了Resp-229k基准数据集,包含22.9万条带大语言模型提炼临床叙述的录音。大量实验表明,Resp-Agent在多种评估设置下均优于现有方法,显著提升数据稀缺和长尾类别不平衡条件下的诊断鲁棒性。代码与数据已开源。
原文摘要 · Abstract (English)
Deep learning-based respiratory auscultation is currently hindered by two fundamental challenges: (i) inherent information loss, as converting signals into spectrograms discards transient acoustic events and clinical context; (ii) limited data availability, exacerbated by severe class imbalance. To bridge these gaps, we present Resp-Agent, an autonomous multimodal system orchestrated by a novel Active Adversarial Curriculum Agent (Thinker-A$^2$CA). Unlike static pipelines, Thinker-A$^2$CA serves as a central controller that actively identifies diagnostic weaknesses and schedules targeted synthesis in a closed loop. To address the representation gap, we introduce a modality-weaving Diagnoser that weaves clinical text with audio tokens via strategic global attention and sparse audio anchors, capturing both long-range clinical context and millisecond-level transients. To address the data gap, we design a flow matching Generator that adapts a text-only Large Language Model (LLM) via modality injection, decoupling pathological content from acoustic style to synthesize hard-to-diagnose samples. As a foundation for this work, we introduce Resp-229k, a benchmark corpus of 229k recordings paired with LLM-distilled clinical narratives. Extensive experiments demonstrate that Resp-Agent consistently outperforms prior approaches across diverse evaluation settings, improving diagnostic robustness under data scarcity and long-tailed class imbalance. Our code and data are available at https://github.com/zpforlove/Resp-Agent.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。