开发印度多语言孕产妇健康聊天机器人,提升低资源地区医疗可及性。
Developing and evaluating a chatbot to support maternal health care
- 分阶段智能分流+混合检索+证据条件生成,应对复杂查询。
- 紧急情况召回率达86.7%,在误报与漏报间实现平衡。
- 适合医疗科技、公共健康与跨语言AI研究者参考。
通过手机聊天机器人提供可信的孕产妇健康信息,在健康素养低、医疗资源有限的地区具有重要意义。但实际部署面临挑战:用户提问简短模糊、语言混杂,回答需结合本地化背景,且症状信息不全使安全转诊困难。本文介绍一个由学术界、健康科技公司、非营利组织与医院合作开发的印度孕产妇健康聊天机器人。系统融合(1)阶段感知的分诊机制,将高风险问题导向专家模板;(2)基于精选孕婴指南的混合检索;(3)基于大模型的证据条件生成。核心贡献是面向高风险部署的多方法评估流程:(i)标注的分诊基准(N=150),紧急情况召回率86.7%,明确报告漏报与误报权衡;(ii)合成的多证据检索基准(N=100),含段落级证据标签;(iii)使用临床医生设计标准对真实查询进行大模型作为评判者的对比测试(N=781);(iv)专家验证。结果表明,可信医疗助手需深度防御设计与多方法评估相结合,而非依赖单一模型或评估方式。
原文摘要 · Abstract (English)
The ability to provide trustworthy maternal health information using phone-based chatbots can have a significant impact, particularly in low-resource settings where users have low health literacy and limited access to care. However, deploying such systems is technically challenging: user queries are short, underspecified, and code-mixed across languages, answers require regional context-specific grounding, and partial or missing symptom context makes safe routing decisions difficult. We present a chatbot for maternal health in India developed through a partnership between academic researchers, a health tech company, a public health nonprofit, and a hospital. The system combines (1) stage-aware triage, routing high-risk queries to expert templates, (2) hybrid retrieval over curated maternal/newborn guidelines, and (3) evidence-conditioned generation from an LLM. Our core contribution is an evaluation workflow for high-stakes deployment under limited expert supervision. Targeting both component-level and end-to-end testing, we introduce: (i) a labeled triage benchmark (N=150) achieving 86.7% emergency recall, explicitly reporting the missed-emergency vs. over-escalation trade-off; (ii) a synthetic multi-evidence retrieval benchmark (N=100) with chunk-level evidence labels; (iii) LLM-as-judge comparison on real queries (N=781) using clinician-codesigned criteria; and (iv) expert validation. Our findings show that trustworthy medical assistants in multilingual, noisy settings require defense-in-depth design paired with multi-method evaluation, rather than any single model and evaluation method choice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。