用大模型自动匹配医疗编码，准确率超85%。
Large Language Models as Medical Codes Selectors: a benchmark using the International Classification of Primary Care
- 用检索增强+大模型选码，不需微调即可达高准确率。
- 33个模型中28个F1超0.8，最优达0.85以上。
- 适合医疗数据标准化、临床研究自动化场景。
背景：医疗编码用于结构化医疗数据，支持研究、质量监控与政策制定。本研究评估大语言模型（LLM）在使用领域特定搜索引擎输出的基础上，为巴西葡萄牙语临床表述分配ICPC-2编码的潜力。方法：采用包含437条临床表达的数据集，每条标注有ICPC-2编码；通过OpenAI text-embedding-3-large检索73,563个已标注概念作为候选。33个LLM对每个查询进行提示并选择最佳匹配的ICPC-2编码。评估指标包括F1分数，以及令牌消耗、成本、响应时间与格式一致性。结果：28个模型的F1得分超过0.8，10个超过0.85，表现最优者为gpt-4.5-preview、o3和gemini-2.5-pro。检索器优化可使性能提升最多4个百分点。多数模型返回符合预期格式的有效编码，幻觉现象减少。小于30亿参数的小模型在格式和输入长度上表现不佳。结论：大模型在无需微调的情况下展现出自动化分配ICPC-2编码的强大潜力。本研究提供基准，但结果受限于数据集范围与实验设置。未来需开展更广泛、多语言、端到端的临床验证。
原文摘要 · Abstract (English)
Background: Medical coding structures healthcare data for research, quality monitoring, and policy. This study assesses the potential of large language models (LLMs) to assign ICPC-2 codes using the output of a domain-specific search engine. Methods: A dataset of 437 Brazilian Portuguese clinical expressions, each annotated with ICPC-2 codes, was used. A semantic search engine (OpenAI's text-embedding-3-large) retrieved candidates from 73,563 labeled concepts. Thirty-three LLMs were prompted with each query and retrieved results to select the best-matching ICPC-2 code. Performance was evaluated using F1-score, along with token usage, cost, response time, and format adherence. Results: Twenty-eight models achieved F1-score > 0.8; ten exceeded 0.85. Top performers included gpt-4.5-preview, o3, and gemini-2.5-pro. Retriever optimization can improve performance by up to 4 points. Most models returned valid codes in the expected format, with reduced hallucinations. Smaller models (<3B) struggled with formatting and input length. Conclusions: LLMs show strong potential for automating ICPC-2 coding, even without fine-tuning. This work offers a benchmark and highlights challenges, but findings are limited by dataset scope and setup. Broader, multilingual, end-to-end evaluations are needed for clinical validation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。