用真实生物信息学数据微调大模型，让它们更懂专业术语和实际问题。
An Empirical Analysis of Fine-Tuning Large Language Models on Bioinformatics Literature: PRSGPT and BioStarsGPT
- 构建九步可复现流程，用论坛问答和工具文档训练专用模型
- 微调后模型在专业任务中准确率达61.9%，比肩Gemini但解释更详细
- 开源超18万组问答数据，适合想本地部署生物信息助手的研究者
大型语言模型在复杂生物信息学应用中常缺乏专业知识。本文提出一个可复现的微调流程，用于在特定生物信息学数据上训练大模型，通过两个案例验证：聚焦多基因风险评分（PRS）工具的PRSGPT，以及基于社区论坛讨论训练的BioStarsGPT。该九步流程整合多种数据源，包含结构化预处理、基于提示的问答生成（使用Google Gemini）、自然语言推理（NLI）质量控制、语义去重、基于聚类的数据划分，以及使用LoRA的参数高效微调。我们对三个模型（LLaMA-3.2-3B、Qwen2.5-7B、Gemma）进行微调，并在超过14项词汇与语义指标上进行评测。Qwen2.5-7B表现最佳，其PRSGPT在BLEU-4和ROUGE-1上分别提升82%和70%，BioStarsGPT分别提升6%和18%。生成的开源数据集包含超过28,000个针对PRSGPT的问答对和154,282个针对BioStarsGPT的问答对。人类评估显示，PRSGPT在比较PRS工具任务中的准确率为61.9%，与Google Gemini（61.4%）相当，但提供更丰富的方法细节和准确引用。BioStarsGPT在142个精心筛选的生物信息学问题中达到59%的概念准确率。该流程支持可扩展的领域专用微调，实现隐私保护、本地部署的生物信息学助手，探索其实际应用，并解决开发与使用中的挑战、局限及缓解策略。
原文摘要 · Abstract (English)
Large language models (LLMs) often lack specialized knowledge for complex bioinformatics applications. We present a reproducible pipeline for fine-tuning LLMs on specialized bioinformatics data, demonstrated through two use cases: PRSGPT, focused on polygenic risk score (PRS) tools, and BioStarsGPT, trained on community forum discussions. The nine-step pipeline integrates diverse data sources, structured preprocessing, prompt-based question-answer (QA) generation (via Google Gemini), natural language inference (NLI) for quality control, semantic deduplication, clustering-based data splitting, and parameter-efficient fine-tuning using LoRA. We fine-tuned three LLMs (LLaMA-3.2-3B, Qwen2.5-7B, Gemma) and benchmarked them on over 14 lexical and semantic metrics. Qwen2.5-7B emerged as the best performer, with BLEU-4 and ROUGE-1 improvements of 82\% and 70\% for PRSGPT and 6\% and 18\% for BioStarsGPT, respectively. The open-source datasets produced include over 28,000 QA pairs for PRSGPT and 154,282 for BioStarsGPT. Human evaluation of PRSGPT yielded 61.9\% accuracy on the PRS tools comparison task, comparable to Google Gemini (61.4\%), but with richer methodological detail and accurate citations. BioStarsGPT demonstrated 59\% conceptual accuracy across 142 curated bioinformatics questions. Our pipeline enables scalable, domain-specific fine-tuning of LLMs. It enables privacy-preserving, locally deployable bioinformatics assistants, explores their practical applications, and addresses the challenges, limitations, and mitigation strategies associated with their development and use.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。