大模型靠作者信息预测学术合作,比传统方法更懂‘谁可能合著’。
Can LLMs Predict Academic Collaboration? Topology Heuristics vs. LLM-Based Link Prediction on Real Co-authorship Networks
- 用作者资料而非图结构,让大模型推理潜在合作者
- 在无共同作者的新合作预测中,大模型准确率达71.4%-78.9%(AUROC)
- 适合研究合作模式、跨领域创新的学者参考
大语言模型能否预测学术合作?我们基于OpenAlex数据集(996万作者,1087万条边)开展链接预测研究,评估仅凭作者档案预测未来合作的能力。使用Qwen2.5-72B-Instruct在三个AI研究历史时期进行实验,发现大模型与拓扑启发式方法捕捉不同信号且互补:在自然类别不平衡条件下,大模型的AUROC达0.714–0.789,优于共同邻居、杰卡德、偏好连接等方法,召回率最高达92.9%;在平衡评估下,大模型在所有时期均超越所有拓扑方法(AUROC 0.601–0.658 vs. 最优启发式0.525–0.538);对持续合作,大模型表现(0.687)接近Adamic-Adar(0.684)。关键发现:78.6%–82.7%的新合作发生在无共同邻居的作者之间——拓扑方法全失效,但大模型仍实现AUROC 0.652,依靠作者元数据推理。时间维度消融显示,研究概念是主导信号(移除后AUROC下降0.047–0.084)。预计算图特征反而降低性能,因锚定效应,表明大模型与拓扑方法应独立运作。社会文化消融显示,姓名推断的族裔与机构国家无法超越拓扑信号,反映人工智能领域的同质性。node2vec基线表现接近Adamic-Adar,说明大模型获取的是作者元数据这一根本不同的信息通道。
原文摘要 · Abstract (English)
Can large language models (LLMs) predict which researchers will collaborate? We study this question through link prediction on real-world co-authorship networks from OpenAlex (9.96M authors, 108.7M edges), evaluating whether LLMs can predict future scientific collaborations using only author profiles, without access to graph structure. Using Qwen2.5-72B-Instruct across three historical eras of AI research, we find that LLMs and topology heuristics capture distinct signals and are strongest in complementary settings. On new-edge prediction under natural class imbalance, the LLM achieves AUROC 0.714--0.789, outperforming Common Neighbors, Jaccard, and Preferential Attachment, with recall up to 92.9\%; under balanced evaluation, the LLM outperforms \emph{all} topology heuristics in every era (AUROC 0.601--0.658 vs.\ best-heuristic 0.525--0.538); on continued edges, the LLM (0.687) is competitive with Adamic-Adar (0.684). Critically, 78.6--82.7\% of new collaborations occur between authors with no common neighbor -- a blind spot where all topology heuristics score zero but the LLM still achieves AUROC 0.652 by reasoning from author metadata alone. A temporal metadata ablation reveals that research concepts are the dominant signal (removing concepts drops AUROC by 0.047--0.084). Providing pre-computed graph features to the LLM \emph{degrades} performance due to anchoring effects, confirming that LLMs and topology methods should operate as separate, complementary channels. A socio-cultural ablation finds that name-inferred ethnicity and institutional country do not predict collaboration beyond topology, reflecting the demographic homogeneity of AI research. A node2vec baseline achieves AUROC comparable to Adamic-Adar, establishing that LLMs access a fundamentally different information channel -- author metadata -- rather than encoding the same structural signal differently.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。