小规模开源大模型在生物医学问答中已逼近甚至超越闭源模型。
Are Smaller Open-Weight LLMs Closing the Gap to Proprietary Models for Biomedical Question Answering?
- 用检索+上下文学习+结构化输出提升小模型问答能力
- 集成多个开源模型后，部分任务超越GPT-4o等闭源模型
- 适合关注开源AI在医疗领域应用的研究者与开发者
开源大语言模型（LLM）快速进步，如DeepSeek-V3已达到与闭源模型相当的性能。本文聚焦生物医学问答领域，参与BioASQ挑战赛第13B阶段任务。对比了多个开源模型与GPT-4o、GPT-4.1、Claude 3.5 Sonnet及Claude 3.7 Sonnet等领先系统。通过基于嵌入距离的片段检索、上下文学习和结构化输出等技术增强问答能力，并对特定任务采用模型集成策略以融合不同模型输出。结果表明，开源模型表现可媲美甚至超过闭源模型，尤其在集成策略下优势显著。所有代码已公开于https://github.com/evidenceprime/BioASQ-13b。
原文摘要 · Abstract (English)
Open-weight versions of large language models (LLMs) are rapidly advancing, with state-of-the-art models like DeepSeek-V3 now performing comparably to proprietary LLMs. This progression raises the question of whether small open-weight LLMs are capable of effectively replacing larger closed-source models. We are particularly interested in the context of biomedical question-answering, a domain we explored by participating in Task 13B Phase B of the BioASQ challenge. In this work, we compare several open-weight models against top-performing systems such as GPT-4o, GPT-4.1, Claude 3.5 Sonnet, and Claude 3.7 Sonnet. To enhance question answering capabilities, we use various techniques including retrieving the most relevant snippets based on embedding distance, in-context learning, and structured outputs. For certain submissions, we utilize ensemble approaches to leverage the diverse outputs generated by different models for exact-answer questions. Our results demonstrate that open-weight LLMs are comparable to proprietary ones. In some instances, open-weight LLMs even surpassed their closed counterparts, particularly when ensembling strategies were applied. All code is publicly available at https://github.com/evidenceprime/BioASQ-13b.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。