大模型研究投入越多,影响力未必越大,资源集中不等于成果出彩。
More Computational Resources Do Not Ensure Higher Scholarly Impact: Evidence from Leading NLP Conference Papers

- 分析1.39万篇顶会论文,用GPU型号和数量衡量算力投入
- 顶尖20%论文占89%算力却仅获32%引用和33%奖项
- 算力提升对影响力影响微弱,更多靠论文本身质量
计算资源在NLP研究中日益重要,但其与学术影响力的关联仍不清晰。我们分析了2020至2025年间13,921篇ACL、EMNLP和NAACL主会论文,以GPU配置作为计算资源的量化指标。从全文提取GPU型号与数量,将每篇论文最大配置标准化为可比硬件能力值,并关联引用数、奖项、主题与机构信息。尽管GPU报告率上升,仍不完整;算力增长主要来自新硬件代际和中等规模多卡配置。资源集中度远超影响力集中度:年度前20%的论文占83.9%-89.9%的算力,但仅获得27%-32%的引用和20%-33%的论文奖项。在调整模型中,算力总量增加十倍,对应引用百分位提升3.52个百分点,模型R²仅提高0.0042。相比新硬件代际,GPU数量与引用及获奖更一致相关。总体而言,报告算力虽与影响力有关,但难以独立解释研究影响力。
原文摘要 · Abstract (English)
Computational resources are increasingly central to NLP research, but how closely reported GPU capability aligns with scholarly impact remains unclear. We analyze 13,921 ACL, EMNLP, and NAACL main-conference papers published between 2020 and 2025, using GPU resources as our operational measure of computational resources. From full texts, we extract GPU models and counts, standardize each paper's largest reported configuration into a comparable hardware-capability measure, and link these data to citation, award, topic, and institutional metadata. GPU reporting became more common but remained incomplete, while reported capability increased mainly through newer hardware generations and medium-scale multi-GPU configurations. Resource concentration substantially exceeded impact concentration: the annual top 20% of GPU-quantifiable papers accounted for 83.9%-89.9% of reported GPU capability, but only 27%-32% of citations and 20%-33% of paper awards. In adjusted models, a tenfold increase in aggregate reported GPU capability was associated with a 3.52-percentage-point increase in within-NLP topic-year citation percentile, but increased model R^2 by only 0.0042. GPU count showed more consistent positive associations with citation and award outcomes than newer hardware generation. Overall, reported GPU resources are associated with scholarly impact but provide little standalone explanation of research influence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。