让对话系统实时识别咳嗽类型,辅助远程医疗判断病情。
From Sound to Symptom: Real-Time Respiratory Signal Understanding for Conversational Healthcare Agents
- 用滚动音频缓冲区同步对话轮次,实现毫秒级呼吸信号检测
- 可区分干咳湿咳等类型,准确率达93%且延迟仅340ms
- 结合对话上下文智能过滤误报,适合临床医生在远程问诊中使用
说话过程中的咳嗽事件包含有价值的呼吸信号,但现有对话系统将其视为噪声丢弃。我们提出 HealthCUES(基于具身声音的临床理解),一个面向实时对话代理的非语言呼吸信号监测流式处理管道,据我们所知,这是首个具备该能力的系统。HealthCUES 通过与对话轮次对齐的滚动缓冲区处理音频,实现亚秒级事件检测而不打断对话流程。除二分类咳嗽检测外,系统还提供细粒度分析:(i) 区分咳嗽与清嗓,(ii) 咳嗽亚型分类(干咳、湿咳、犬吠样、百日咳样)并输出置信度分数,(iii) 估算持续时间及起止边界。为避免警报疲劳,系统引入对话感知的触发抑制机制。采用 Qwen3Omni 多模态大模型,通过约束结构化输出将咳嗽分析分解为独立提示优化的并行任务。在 847 段自建对话音频上的评估显示,咳嗽检测 F1 达 93%,湿咳/干咳亚型分类加权 F1 为 0.75,端到端平均延迟 340ms;在 AMI 会议语料库上的外部验证确认了系统在有语音干扰下对咳嗽、清嗓和语音分离的鲁棒性(宏平均 F1 0.91)。由持证医疗专业人员参与的用户研究证实亚型信息具有临床意义,且系统适用于远程医疗工作流程。
原文摘要 · Abstract (English)
Cough events during live spoken conversations carry clinically valuable respiratory signals, yet existing dialogue systems treat them as acoustic noise to be discarded. We present HealthCUES (Clinical Understanding from Embodied Sounds), a streaming pipeline for paralinguistic respiratory monitoring in real-time conversational agents, a capability that, to the best of our knowledge, is absent from all prior systems. HealthCUES processes audio through a rolling buffer aligned with dialogue turn boundaries, enabling sub-second event detection without interrupting conversational flow. Beyond binary cough detection, the system provides fine-grained analytics: (i) differentiation between coughing and throat clearing, (ii) cough subtype classification (dry, wet, barking, whooping) with confidence scores, and (iii) temporal duration estimation with start-end boundaries. To prevent alert fatigue, HealthCUES introduces dialogue-aware gating mechanisms that modulate triggering based on conversational context. The system leverages Qwen3Omni, a multimodal large language model (MLLM), with constrained structured outputs, decomposing cough analysis into parallel prediction tasks for independent prompt optimization. Evaluation on 847 in-house conversational audio segments demonstrates 93\% F1 for cough detection, 0.75 weighted-F1 for wet/dry subtype classification, and average end-to-end latency of 340ms; external validation on the AMI meeting corpus confirms robust cough, throat-clearing, and speech separation in the presence of speech (0.91 macro-F1). A user study with licensed healthcare professionals confirms the clinical relevance of subtype information and the system's utility in telehealth workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。