让大模型写论文时能自动精准引用文献,提升学术写作质量。
ScholarCopilot: Training Large Language Models for Academic Writing with Accurate Citations
- 用[RET]标记触发文献检索,动态结合引用数据库增强生成
- 在500K篇arXiv论文上训练,顶1检索准确率达40.1%
- 7B小模型胜过更大模型,适合科研人员高效写作
学术写作需要连贯的文本生成和精确的文献引用。尽管最近的检索增强生成(RAG)系统显著提升了通用文本生成的事实准确性,但其对专业学术写作的支持仍有限。本文提出ScholarCopilot,一个统一框架,旨在提升现有大语言模型生成专业学术文章的能力,确保引文准确且上下文相关。该框架通过生成检索标记[RET]动态决定何时检索学术参考文献,并查询引用数据库获取信息。被检索到的参考文献随后输入模型以增强生成过程。我们在单一框架中联合优化生成与引文任务,提高效率。模型基于Qwen-2.5-7B,在50万篇arXiv论文上训练。在评估数据集上,其顶1检索准确率达40.1%,优于E5-Mistral-7B-Instruct(15.0%)和BM25(9.8%)。在1,000个学术写作样本数据集上,ScholarCopilot生成质量评分为16.2/25(涵盖相关性、连贯性、学术严谨性、完整性与创新性),显著超越现有模型,包括更大的Retrieval-Augmented Qwen2.5-72B-Instruct。人工评估显示,尽管是7B模型,ScholarCopilot在引文质量上获得100%偏好,整体有用性超过70%。
原文摘要 · Abstract (English)
Academic writing requires both coherent text generation and precise citation of relevant literature. Although recent Retrieval-Augmented Generation (RAG) systems have significantly improved factual accuracy in general-purpose text generation, their ability to support professional academic writing remains limited. In this work, we introduce ScholarCopilot, a unified framework designed to enhance existing large language models for generating professional academic articles with accurate and contextually relevant citations. ScholarCopilot dynamically determines when to retrieve scholarly references by generating a retrieval token [RET], which is then used to query a citation database. The retrieved references are fed into the model to augment the generation process. We jointly optimize both the generation and citation tasks within a single framework to improve efficiency. Our model is built upon Qwen-2.5-7B and trained on 500K papers from arXiv. It achieves a top-1 retrieval accuracy of 40.1% on our evaluation dataset, outperforming baselines such as E5-Mistral-7B-Instruct (15.0%) and BM25 (9.8%). On a dataset of 1,000 academic writing samples, ScholarCopilot scores 16.2/25 in generation quality -- measured across relevance, coherence, academic rigor, completeness, and innovation -- significantly surpassing all existing models, including much larger ones like the Retrieval-Augmented Qwen2.5-72B-Instruct. Human studies further demonstrate that ScholarCopilot, despite being a 7B model, significantly outperforms ChatGPT, achieving 100% preference in citation quality and over 70% in overall usefulness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。