用动态轨迹建模预测论文影响力,比纯文本评估更准。
FAME: Forecasting Academic Impact via Continuous-Time Manifold Evolution

- 构建连续时间流形演化模型,融合文本与知识传播路径
- 在3200篇论文上预测效果显著优于现有大模型
- 适合需要量化科研潜力评估的机构或审稿人
大语言模型被广泛用于科研创意的构思与评估,但其判断难以验证,因真正影响可能需数年才显现。本文以人类撰写论文的影响预测为可验证任务,发现前沿大模型无法可靠区分高影响力与普通论文,表明静态文本判断不足。为此提出FAME(基于连续时间流形演化的学术影响力预测),通过时空框架建模科学主题的动态演变轨迹。该模型将论文映射到由文本特征和已验证知识流图驱动的动态潜在空间,学习几何约束,使高影响力论文与领域前进动量对齐。在三个快速演进子领域的3200篇arXiv论文上,FAME在前瞻性多维影响力预测中持续显著超越现有先进大模型。此外,将FAME的动态几何信号融入大模型后,其预测性能大幅提升。结果支持将论文影响力预测作为可度量的代理基准,并确立FAME作为自动科研评估的轨迹感知基础。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly used to brainstorm and evaluate research ideas, yet assessing such judgments is fundamentally difficult because the true impact of a new idea may take years to emerge. We address this challenge by using the impact forecasting of human-authored manuscripts as a verifiable proxy task. In a prospective forecasting study, we find that frontier LLMs fail to reliably distinguish high-impact papers from ordinary publications, suggesting that static text-based judging is insufficient for scientific evaluation. To address this limitation, we propose $\textbf{FAME}$ ($\underline{\text{F}}$orecasting $\underline{\text{A}}$cademic Impact via Continuous-Time $\underline{\text{M}}$anifold $\underline{\text{E}}$volution), a spatiotemporal framework for modeling the dynamic trajectories of scientific topics. FAME projects papers into a dynamic latent space informed by textual features and a verified knowledge-flow graph, learning geometric constraints that align impactful manuscripts with the forward momentum of their fields. Experiments on 3,200 arXiv papers across three fast-evolving subfields show that FAME consistently and substantially outperforms state-of-the-art LLM evaluators in prospective multidimensional impact forecasting. Furthermore, integrating FAME's dynamic geometric signals into LLMs significantly improves their forecasting performance. These results support manuscript impact forecasting as a useful, measurable proxy benchmark and position FAME as a strong, trajectory-aware foundation for automated scientific evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。