研究发现大模型正深度渗透学术写作与评审流程。
Large Language Models Penetration in Scholarly Writing and Peer Review
- 构建双工具框架，分别用于内容采集与大模型使用检测
- 实验证明大模型在学术流程中应用比例持续上升
- 适合关注学术伦理与可信度的研究者参考
尽管大型语言模型（LLMs）的广泛应用带来便利，也引发了对学术研究可信度和学术流程可靠性的担忧。为深入理解这一趋势，我们从多个视角和维度评估了大模型在学术工作流中的渗透程度，提供了有力证据表明其影响日益显著。本文提出一个包含两个组件的框架： exttt{ScholarLens}，一个涵盖人类与大模型生成内容的学术写作与同行评审的精选数据集，支持多视角评估； exttt{LLMetrica}，一套基于规则的指标与基于模型的检测器相结合的工具，实现多维度评估大模型渗透率。实验结果验证了 exttt{LLMetrica}的有效性，揭示了大模型在学术流程中作用不断增强的现实。研究强调，在大模型使用中必须加强透明度、责任追究与伦理规范，以维护学术公信力。
原文摘要 · Abstract (English)
While the widespread use of Large Language Models (LLMs) brings convenience, it also raises concerns about the credibility of academic research and scholarly processes. To better understand these dynamics, we evaluate the penetration of LLMs across academic workflows from multiple perspectives and dimensions, providing compelling evidence of their growing influence. We propose a framework with two components: \texttt{ScholarLens}, a curated dataset of human- and LLM-generated content across scholarly writing and peer review for multi-perspective evaluation, and \texttt{LLMetrica}, a tool for assessing LLM penetration using rule-based metrics and model-based detectors for multi-dimensional evaluation. Our experiments demonstrate the effectiveness of \texttt{LLMetrica}, revealing the increasing role of LLMs in scholarly processes. These findings emphasize the need for transparency, accountability, and ethical practices in LLM usage to maintain academic credibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。