用少量资源让小模型学会医学推理，效果不降反升
Resource-Efficient Fine-Tuning of LLaMA-3.2-3B for Medical Chain-of-Thought Reasoning
- 用LoRA/QLoRA等轻量方法微调，节省60%显存
- 在医疗问答任务中推理更连贯、事实更准确
- 适合算力有限的医院或研究机构使用
大型语言模型如GPT-4和LLaMA展现出强大的推理能力，但微调需大量计算资源。本文提出一种资源高效的微调方法，针对LLaMA-3.2-3B模型提升其在医学链式思维推理方面的能力，同时在受限的GPU和内存环境下运行。通过参数高效微调技术（如LoRA和QLoRA），在公开的医疗推理数据集上对基础模型进行适配。实验表明，该方法在保持强推理能力的同时，相比标准全量微调可减少高达60%的内存占用。评估结果显示，轻量化调整仍能有效提升医疗问答任务中的推理连贯性和事实准确性。本工作为低资源研究环境部署LLM提供了实用策略，并为医疗AI系统在效率与领域专精间的平衡提供启示。
原文摘要 · Abstract (English)
Large Language Models (LLMs) such as GPT-4 and LLaMA have demonstrated remarkable reasoning abilities but require significant computational resources for fine-tuning. This paper presents a resource-efficient fine-tuning approach for LLaMA-3.2-3B to enhance medical chain-of-thought reasoning while operating under constrained GPU and memory settings. Using parameter-efficient tuning techniques such as LoRA and QLoRA, we adapt the base model on publicly available medical reasoning datasets. The model achieves improved reasoning coherence and factual accuracy while reducing memory usage by up to 60% compared to standard full fine-tuning. Experimental evaluation demonstrates that lightweight adaptations can retain strong reasoning capability in medical question-answering tasks. This work highlights practical strategies for deploying LLMs in low-resource research environments and provides insights into balancing efficiency and domain specialization for medical AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。