用思维树框架提升量化医学模型推理能力,准确率最高提升15%。
QM-ToT: A Medical Tree of Thoughts Reasoning Framework for Quantized Model
- 基于思维树分解医学难题,分步推理并评估路径
- INT4量化下,70b模型准确率从34%提至50%,8b模型从58.77%提至69.49%
- 仅用3.9%数据实现86.27%的蒸馏增益,适合资源受限医疗场景
大语言模型在专业生物医学任务中面临挑战,源于医学推理的复杂性与临床数据的敏感性。现有模型在量化部署时性能下降,难以满足临床需求。为此,我们提出量化医学思维树(QM-ToT)框架,采用基于路径的思维树(ToT)推理方法,将复杂医学问题分解为可管理子任务,并引入评估层。该框架显著提升了INT4量化模型在挑战性MedQAUSMLE数据集上的表现:LLaMA2-70b模型准确率从34%提升至50%,LLaMA-3.1-8b模型从58.77%提升至69.49%。此外,我们提出一种基于ToT的高效数据蒸馏方法,相比传统方法,在仅使用3.9%数据的情况下实现86.27%的性能提升。本工作首次展示思维树在复杂生物医学任务中的巨大潜力,为资源受限医疗环境中高效率量化大模型部署奠定基础。
原文摘要 · Abstract (English)
Large language models (LLMs) face significant challenges in specialized biomedical tasks due to the inherent complexity of medical reasoning and the sensitive nature of clinical data. Existing LLMs often struggle with intricate medical terminology and the need for accurate clinical insights, leading to performance reduction when quantized for resource-constrained deployment. To address these issues, we propose Quantized Medical Tree of Thought (QM-ToT), a path-based reasoning framework. QM-ToT leverages a Tree of Thought (ToT) reasoning approach to decompose complex medical problems into manageable subtasks, coupled with evaluator assessment layers. This framework facilitates substantial performance improvements in INT4-quantized models on the challenging MedQAUSMLE dataset. Specifically, we demonstrate a remarkable accuracy increase from 34% to 50% for the LLaMA2-70b model and from 58.77% to 69.49% for LLaMA-3.1-8b. Besides, we also proposed an effect data distillation method based on ToT. Compared to the traditional distillation method, we achieved an improvement of 86. 27% while using only 3.9% of the data.This work, for the first time, showcases the potential of ToT to significantly enhance performance on complex biomedical tasks, establishing a crucial foundation for future advances in deploying high-performing quantized LLM in resource-limited medical settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。