研究AI写作工具对学术创新的影响,发现非英语国家学者创新力下降更明显。
Research Novelty in Information Systems Journals After ChatGPT: Differences Across Institutional Language Contexts
- 用语义距离衡量论文与近似前作的差异,分析2020-2025年44本顶刊文章
- 非英语主导机构作者的论文创新性在2022年后下降0.176个标准差,约7个百分点
- 揭示生成式AI可能让已有范式更易复制,影响不同语言背景学者的创新平衡
大型语言模型在学术研究中日益普及,但其带来的生产力提升是否伴随研究新颖性的变化尚不明确。本文考察了2022年ChatGPT广泛可用后,信息系统期刊中论文在抽象层面的语义新颖性如何变化,以及这种变化在不同机构语言背景下的差异。分析2020至2025年间44本A*和A类信息科学期刊的13,847篇文章,使用SPECTER2模型表示标题与摘要,计算每篇论文与其最近邻近期论文之间的语义距离,并构建前后对比模型。结果显示,第一作者隶属非英语主导国家机构的文章,在2022年后相对语义新颖性下降0.176个标准差,相当于约7个百分点,显著高于英语主导机构。该模式在多种替代设定下保持一致,尽管联合作者估计的精度较低。研究认为,生成式AI支持的知识工作存在张力:一方面拓宽了对既有知识的访问并支持新组合,另一方面也可能使既有框架更容易被重复。由于未观测到个体LLM使用情况,结果反映的是2022年后异质性变化,而非直接由LLM采用所致。研究通过关注论文的语义定位而非发表数量,扩展了对大模型与学术生产率关系的理解,并揭示了后2022年变化在不同机构语境中的差异。
原文摘要 · Abstract (English)
Large language models are increasingly used in scholarly work, yet it remains unclear whether their productivity gains are accompanied by changes in research novelty. We examine how relative abstract-level semantic novelty in Information Systems journals changed after ChatGPT became widely available and whether this change differed across institutional language contexts. We analyze 13,847 articles published from 2020 to 2025 in 44 A* and A Information Systems journals. Using SPECTER2 representations of titles and abstracts, we measure each article's semantic distance from its nearest recent predecessors and estimate a comparative pre/post model. Articles whose first authors were affiliated with institutions in non-English-dominant countries show a 0.176 standard deviation larger post-2022 decline in relative semantic novelty than articles from English-dominant affiliations, equivalent to about 7 percentile points. The pattern is similar across several alternative specifications, although the balanced-author estimate is less precise. We interpret this finding through a tension in generative AI-supported knowledge work. GenAI can widen access to prior knowledge and support new combinations, but it can also make established frames easier to reproduce. Because individual LLM use is not observed, the result identifies a heterogeneous post-2022 shift rather than an effect of LLM adoption. The study extends research on LLMs and scholarly productivity by shifting attention from publication counts to the semantic positioning of published articles and by showing that post-2022 change differs across institutional contexts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。