用20个细调模型专攻高能理论物理，效果优于通用大模型。
FeynTune: Large Language Models for High-Energy Theory
- 基于Llama-3.1微调，针对hep-th等三类粒子物理论文摘要训练
- 在hep-th摘要补全任务上超越基础模型及主流商业大模型
- 验证了领域专用模型在高能物理中的潜力，适合理论物理研究者
我们为理论高能物理领域构建了20个经过微调的专用大语言模型，基于80亿参数的Llama-3.1模型，每种变体均在hep-th、hep-ph与gr-qc三类arXiv摘要（截至2024年8月）的不同组合数据上训练。为对比，还训练了涵盖q-bio和cs领域的模型。所有模型采用两种低秩适配（LoRA）方法，并在不同数据规模下微调。结果表明，这些专用模型在hep-th摘要补全任务中均优于基线模型。我们将其性能与ChatGPT、Claude、Gemini、DeepSeek等主流商用大模型进行比较，揭示了为高能理论物理定制语言模型的有效性与发展方向。
原文摘要 · Abstract (English)
We present specialized Large Language Models for theoretical High-Energy Physics, obtained as 20 fine-tuned variants of the 8-billion parameter Llama-3.1 model. Each variant was trained on arXiv abstracts (through August 2024) from different combinations of hep-th, hep-ph and gr-qc. For a comparative study, we also trained models on datasets that contained abstracts from disparate fields such as the q-bio and cs categories. All models were fine-tuned using two distinct Low-Rank Adaptation fine-tuning approaches and varying dataset sizes, and outperformed the base model on hep-th abstract completion tasks. We compare performance against leading commercial LLMs (ChatGPT, Claude, Gemini, DeepSeek) and derive insights for further developing specialized language models for High-Energy Theoretical Physics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。