arXiv:2510.20279cs.LG2025-10被引 2

用高质量数据训练的AI能成科研好帮手,7B模型也能超越大厂闭源模型。

ResearchGPT: Benchmarking and Training LLMs for End-to-End Computer Science Research Workflows

  • 构建论文级问答数据集,让AI从读论文到写研究全程受训。
  • 7B小模型经专业训练后,性能超GPT-4o、Gemini 2.5 Pro等大模型。
  • 适合想打造领域专用科研助手的研究者与开发者参考。

随着大语言模型发展,科学界期待构建一个能全程协助人类科研的AI合作者,称为ResearchGPT。实现此愿景需评估端到端科研流程的基准,而非孤立任务。为此,我们构建了CS-54k——一个来自1.4万篇开源论文的高质量计算机科学问答语料库,通过检索增强生成与多阶段质量控制确保事实准确。从中衍生出两个子集:用于评估的CS-4k(4k样本)和用于训练的CS-50k(50k样本)。实验表明,CS-4k能有效区分主流大模型的能力层级。在监督学习与强化学习下,使用CS-50k训练的开源模型表现显著提升;即使7B规模的模型,经适当训练后也超越多个更大的专有模型,如GPT-4.1、GPT-4o和Gemini 2.5 Pro。这说明,打造高效科研助手的关键在于领域对齐的数据训练,而非单纯依赖预训练规模或通用基准表现。我们公开发布CS-4k与CS-50k,旨在推动人工智能成为计算机科学研究中可靠的合作伙伴。

原文摘要 · Abstract (English)

As large language models (LLMs) advance, the ultimate vision for their role in science is emerging: we could build an AI collaborator to effectively assist human beings throughout the entire scientific research process. We refer to this envisioned system as ResearchGPT. Given that scientific research progresses through multiple interdependent phases, achieving this vision requires rigorous benchmarks that evaluate the end-to-end workflow rather than isolated sub-tasks. To this end, we contribute CS-54k, a high-quality corpus of scientific Q&A pairs in computer science, built from 14k CC-licensed papers. It is constructed through a scalable, paper-grounded pipeline that combines retrieval-augmented generation (RAG) with multi-stage quality control to ensure factual grounding. From this unified corpus, we derive two complementary subsets: CS-4k, a carefully curated benchmark for evaluating AI's ability to assist scientific research, and CS-50k, a large-scale training dataset. Extensive experiments demonstrate that CS-4k stratifies state-of-the-art LLMs into distinct capability tiers. Open models trained on CS-50k with supervised training and reinforcement learning demonstrate substantial improvements. Even 7B-scale models, when properly trained, outperform many larger proprietary systems, such as GPT-4.1, GPT-4o, and Gemini 2.5 Pro. This indicates that making AI models better research assistants relies more on domain-aligned training with high-quality data than on pretraining scale or general benchmark performance. We release CS-4k and CS-50k in the hope of fostering AI systems as reliable collaborators in CS research.

科研AI大模型训练代码生成数据集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。