用智能体系统自动分析医患对话,准确率达92.8%。
An Agentic AI System for Multi-Framework Communication Coding
- 构建四代理系统,分步完成对话编码任务
- 测试集上整体F1达0.928,风湿科表现最佳
- 适合医疗对话分析与临床研究者使用
临床沟通对患者预后至关重要,但大规模人工标注医患对话仍耗时费力、标准不一。现有基于大模型的方法多依赖单任务模型,缺乏跨框架与领域的适应性、可解释性与可靠性。本文提出基于LangGraph的多框架结构化智能体系统MOSAIC,包含计划、更新、标注与验证四个核心代理:计划代理选择编码本并规划流程,更新代理维护实时检索数据库,标注代理采用动态少样本提示的RAG方法进行编码,验证代理提供一致性检查与反馈。在26份训练与50份测试标注语料(涵盖风湿科与妇产科)上评估,MOSAIC测试集整体F1得分为0.928,风湿科子集达0.962,尤其在患者行为类编码(如提问、表达偏好、主张立场)表现突出。消融实验表明其优于基线模型。
原文摘要 · Abstract (English)
Clinical communication is central to patient outcomes, yet large-scale human annotation of patient-provider conversation remains labor-intensive, inconsistent, and difficult to scale. Existing approaches based on large language models typically rely on single-task models that lack adaptability, interpretability, and reliability, especially when applied across various communication frameworks and clinical domains. In this study, we developed a Multi-framework Structured Agentic AI system for Clinical Communication (MOSAIC), built on a LangGraph-based architecture that orchestrates four core agents, including a Plan Agent for codebook selection and workflow planning, an Update Agent for maintaining up-to-date retrieval databases, a set of Annotation Agents that applies codebook-guided retrieval-augmented generation (RAG) with dynamic few-shot prompting, and a Verification Agent that provides consistency checks and feedback. To evaluate performance, we compared MOSAIC outputs against gold-standard annotations created by trained human coders. We developed and evaluated MOSAIC using 26 gold standard annotated transcripts for training and 50 transcripts for testing, spanning rheumatology and OB/GYN domains. On the test set, MOSAIC achieved an overall F1 score of 0.928. Performance was highest in the Rheumatology subset (F1 = 0.962) and strongest for Patient Behavior (e.g., patients asking questions, expressing preferences, or showing assertiveness). Ablations revealed that MOSAIC outperforms baseline benchmarking.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。