构建跨19个领域的多维科学影响力评估基准，突破传统引文局限。
SciImpact: A Multi-Dimensional, Multi-Field Benchmark for Scientific Impact Prediction
- 整合引文、奖项、媒体、专利等多源数据，构建21万组对比论文对。
- 小模型经多任务微调后超越大模型和闭源模型，效果显著提升。
- 适合关注科研评价、AI评估与跨领域影响分析的研究者使用。
科学文献的快速增长亟需自动化方法来评估和预测研究影响力。以往工作主要聚焦于引文指标，忽视了对模型在其他影响维度上推理能力的评估。为此，我们提出SciImpact，一个涵盖19个领域的大型多维科学影响力预测基准。该基准通过整合异构数据源与定向网络爬取，捕捉从引文数量到奖项认可、媒体报道、专利引用及成果采纳等多种形式的科学影响力。它包含215,928组对比论文对，反映短期（如最佳论文奖）与长期（如诺贝尔奖）影响力的差异。我们在SciImpact上评估了11种主流大语言模型。结果表明，现成模型在不同维度与领域间表现差异显著；而多任务监督微调使小型模型（如40亿参数）显著优于大型模型（如300亿参数），并超越强大的闭源模型（如o4-mini）。这些结果确立了SciImpact作为挑战性基准的价值，验证了其在多维度、跨领域科学影响力预测中的应用潜力。项目主页：https://flypig23.github.io/sciimpact-homepage/
原文摘要 · Abstract (English)
The rapid growth of scientific literature calls for automated methods to assess and predict research impact. Prior work has largely focused on citation-based metrics, leaving limited evaluation of models' capability to reason about other impact dimensions. To this end, we introduce SciImpact, a large-scale, multi-dimensional benchmark for scientific impact prediction spanning 19 fields. SciImpact captures various forms of scientific influence, ranging from citation counts to award recognition, media attention, patent reference, and artifact adoption, by integrating heterogeneous data sources and targeted web crawling. It comprises 215,928 contrastive paper pairs reflecting meaningful impact differences in both short-term (e.g., Best Paper Award) and long-term settings (e.g., Nobel Prize). We evaluate 11 widely used large language models (LLMs) on SciImpact. Results show that off-the-shelf models exhibit substantial variability across dimensions and fields, while multi-task supervised fine-tuning consistently enables smaller LLMs (e.g., 4B) to markedly outperform much larger models (e.g., 30B) and surpass powerful closed-source LLMs (e.g., o4-mini). These results establish SciImpact as a challenging benchmark and demonstrate its value for multi-dimensional, multi-field scientific impact prediction. Our project homepage is https://flypig23.github.io/sciimpact-homepage/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。