在40亿参数模型上，微调比检索增强更有效提升医学问答准确率。
Domain Fine-Tuning vs. Retrieval-Augmented Generation for Medical Multiple-Choice Question Answering: A Controlled Comparison at the 4B-Parameter Scale
- 固定模型大小与推理流程，对比微调与检索增强两种知识注入方式
- 微调使准确率提升6.8个百分点，达53.3%，显著优于基础模型的46.4%
- 检索增强在任一模型上均未带来统计显著提升，甚至轻微拖累表现
在部署小型开源大语言模型进行医学问答时，开发者常面临抉择：是投入资源对模型进行领域微调，还是保留通用模型并通过检索增强生成（RAG）在推理时注入领域知识。我们通过固定模型规模、提示模板、解码温度、检索流程和评估协议，仅改变两点：(i) 模型是否经过领域适配（Gemma 3 4B 与 MedGemma 4B，均为4比特量化并由Ollama服务）；(ii) 是否在提示中插入来自医学知识库的检索段落。在完整MedQA-USMLE四选一测试集（1,273题）上进行三次重复评估（共15,276次LLM调用）。结果显示，领域微调相比通用4B基线带来6.8个百分点的准确率提升（53.3% vs. 46.4%，McNemar p < 10^-4）。而基于MedMCQA解释的RAG在任一模型上均未产生统计显著增益，在领域微调模型中甚至略有下降（-1.9百分点，p = 0.16）。在该规模与基准下，编码于权重中的领域知识优于通过上下文提供的知识。实验代码与JSONL日志已公开以支持复现。
原文摘要 · Abstract (English)
Practitioners deploying small open-weight large language models (LLMs) for medical question answering face a recurring design choice: invest in a domain-fine-tuned model, or keep a general-purpose model and inject domain knowledge at inference time via retrieval-augmented generation (RAG). We isolate this trade-off by holding model size, prompt template, decoding temperature, retrieval pipeline, and evaluation protocol fixed, and varying only (i) whether the model has been domain-adapted (Gemma 3 4B vs. MedGemma 4B, both 4-bit quantized and served via Ollama) and (ii) whether retrieved passages from a medical knowledge corpus are inserted into the prompt. We evaluate all four cells of this 2x2 design on the full MedQA-USMLE 4-option test split (1,273 questions) with three repetitions per question (15,276 LLM calls). Domain fine-tuning yields a +6.8 percentage-point gain in majority-vote accuracy over the general 4B baseline (53.3% vs. 46.4%, McNemar p < 10^-4). RAG over MedMCQA explanations does not produce a statistically significant gain in either model, and in the domain-tuned model the point estimate is slightly negative (-1.9 pp, p = 0.16). At this scale and on this benchmark, domain knowledge encoded in weights dominates domain knowledge supplied in context. We release the full experiment code and JSONL traces to support replication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。