用精准医学证据库让AI在内分泌专科考试中超越人类和顶级大模型。
Evidence-Grounded Subspecialty Reasoning: Evaluating a Curated Clinical Intelligence Layer on the 2025 Endocrinology Board-Style Examination
- 构建专属内分泌证据库，结合结构化推理生成带出处的答案。
- 在120道专科题中准确率87.5%，最难30题答对76.7%。
- 答案可追溯来源，适合需可解释性的临床AI部署场景。
大型语言模型在通用医学考试中表现优异，但在内分泌等亚专科领域仍面临指南快速更新和证据等级复杂的问题。我们评估了基于证据的临床推理系统January Mirror，在120道内分泌专科考试题上的表现。Mirror整合了经筛选的内分泌与心代谢证据语料库，并采用结构化推理架构生成有证据支持的回答，且在不使用外部检索的封闭环境下运行。对比模型包括具备实时网络访问能力的GPT-5、GPT-5.2和Gemini-3-Pro。结果表明，Mirror准确率达87.5%（105/120；95%置信区间：80.4–92.3%），超过人类参考水平62.3%，并优于各前沿大模型（GPT-5.2: 74.6%，GPT-5: 74.0%，Gemini-3-Pro: 69.8%）。在人类正确率低于50%的30道难题中，其准确率为76.7%。镜像系统的前两名准确率为92.5%，高于GPT-5.2的85.25%。此外，74.2%的输出至少引用一个指南级文献，经人工验证引用准确率达100%。研究证实，经精心构建并标注来源的证据层，可在亚专科临床推理中超越无约束网络检索，支持临床应用中的可审计性。
原文摘要 · Abstract (English)
Background: Large language models have demonstrated strong performance on general medical examinations, but subspecialty clinical reasoning remains challenging due to rapidly evolving guidelines and nuanced evidence hierarchies. Methods: We evaluated January Mirror, an evidence-grounded clinical reasoning system, against frontier LLMs (GPT-5, GPT-5.2, Gemini-3-Pro) on a 120-question endocrinology board-style examination. Mirror integrates a curated endocrinology and cardiometabolic evidence corpus with a structured reasoning architecture to generate evidence-linked outputs. Mirror operated under a closed-evidence constraint without external retrieval. Comparator LLMs had real-time web access to guidelines and primary literature. Results: Mirror achieved 87.5% accuracy (105/120; 95% CI: 80.4-92.3%), exceeding a human reference of 62.3% and frontier LLMs including GPT-5.2 (74.6%), GPT-5 (74.0%), and Gemini-3-Pro (69.8%). On the 30 most difficult questions (human accuracy less than 50%), Mirror achieved 76.7% accuracy. Top-2 accuracy was 92.5% for Mirror versus 85.25% for GPT-5.2. Conclusions: Mirror provided evidence traceability: 74.2% of outputs cited at least one guideline-tier source, with 100% citation accuracy on manual verification. Curated evidence with explicit provenance can outperform unconstrained web retrieval for subspecialty clinical reasoning and supports auditability for clinical deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。