厘清生成式AI在科学计量中的优劣，提出可解释的使用框架。
Generative AI and the future of scientometrics: current topics and future questions
- 区分文本语义与语用维度，构建评估AI应用的理论框架。
- 任务性质、分析粒度和目标类型决定AI使用策略。
- 提醒警惕AI生成文本对科学指标的潜在干扰，需持续研究。
本文参与关于生成式人工智能（GenAI）在科学计量学中应用的讨论。我们认为，从试错走向可解释、可操作的应用，需要系统理解GenAI相较于其他技术及人类判断的优势与局限。为此，我们基于文本语义维度（词语含义）与语用维度（语境嵌入）的区分，提出一个概念性框架，用于解读GenAI在科学计量中的应用结果并指导用户。具体而言，关键考量因素包括任务性质、分析粒度以及目标是描述性、推断性还是评价性，这些因素引导不同的GenAI使用策略与人机协同方式。最后，我们指出，GenAI生成大量科学语言可能改变用于衡量科学活动的文本特征，如作者、词汇和引用模式。因此，未来必须开展严谨的实证研究与理论反思，以持续理解人工智能时代知识生产的演变规律。
原文摘要 · Abstract (English)
In this paper, we contribute to the debate on generative artificial intelligence (GenAI) in scientometrics. We argue that moving from a trial-and-error approach to an explainable and actionable use requires a principled understanding of strengths and weaknesses of GenAI as compared with other techniques and with human judgment. To this end, we introduce a conceptual framework based on the distinction between the semantic dimensions of texts, i.e. the meanings attributed to words, and their pragmatic dimension, i.e. their embedding within communicative situations. We leverage this framework to interpret the results of applications of GenAI in scientometrics and to provide guidance to users. Specifically, we conclude that key parameters to be considered are the nature of the task, the level of granularity of the analysis and whether the goal was descriptive, inferential or evaluative. These parameters lead to different strategies for using GenAI and human-machine integration. Finally, we suggest that, by generating large amounts of scientific language, GenAI might affect textual characteristics used to measure science, such as authors, words, and references. We argue that careful empirical work and theoretical reflection will be essential to remain capable of interpreting the evolving patterns of knowledge production in the age of AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。