用提示工程和轻量微调提升大模型医学问答能力
Instruction Tuning and CoT Prompting for Contextual Medical QA with LLMs
- 对比指令提示与思维链提示，优化医学问答推理
- 轻量微调显著提升准确率，但需注意模型规模影响
- 适合医疗AI研究者和临床辅助系统开发者
大语言模型在医学问答任务中展现出巨大潜力，但因其领域复杂性和标注数据有限，适应生物医学推理仍具挑战。本文研究提示设计与轻量微调对开源LLM在PubMedQA（多选题生物医学问答基准）上的影响。聚焦标准指令提示与思维链（CoT）提示两种策略，并采用QLoRA实现参数高效微调。实验表明，在零样本设置下，仅使用CoT提示即可提升推理能力；而指令微调能显著提高准确率。然而，对部分大型模型而言，在CoT提示上进行微调可能反而降低性能。结果表明，具备推理意识的提示有效，但其收益取决于模型类型与规模。本研究为结合提示工程与高效微调提供了实用指导。
原文摘要 · Abstract (English)
Large language models (LLMs) have shown great potential in medical question answering (MedQA), yet adapting them to biomedical reasoning remains challenging due to domain-specific complexity and limited supervision. In this work, we study how prompt design and lightweight fine-tuning affect the performance of open-source LLMs on PubMedQA, a benchmark for multiple-choice biomedical questions. We focus on two widely used prompting strategies - standard instruction prompts and Chain-of-Thought (CoT) prompts - and apply QLoRA for parameter-efficient instruction tuning. Across multiple model families and sizes, our experiments show that CoT prompting alone can improve reasoning in zero-shot settings, while instruction tuning significantly boosts accuracy. However, fine-tuning on CoT prompts does not universally enhance performance and may even degrade it for certain larger models. These findings suggest that reasoning-aware prompts are useful, but their benefits are model- and scale-dependent. Our study offers practical insights into combining prompt engineering with efficient finetuning for medical QA applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。