arXiv:2511.12529cs.HCcs.AI2025-11被引 4

AI生成的论文摘要经轻微修改后可达到人工水平,关键在是否公开来源。

Accepted with Minor Revisions: Value of AI-Assisted Scientific Writing

  • 通过激励实验对比人类与AI摘要的修改行为,设计双因素对照试验。
  • 无来源时对AI摘要修改最多,公开来源后修改量趋同,接受率与修改次数强相关。
  • 编辑质量比来源更影响评审结果,凸显透明披露的重要性。

大语言模型在多个领域应用广泛,但其作为科学写作辅助工具的有效性——尤其是需要精确性、多模态整合与领域专长的任务——仍不明确。我们研究了LLM在支持领域专家进行科学写作中的潜力,重点关注摘要撰写。设计了一个激励性随机对照试验,采用虚拟会议场景,将具有相关专业知识的参与者分为作者与审稿人两组。受行为科学方法启发,创新的激励机制促使作者将提供的摘要修改至符合同行评审投稿的标准。2×2的被试间设计涵盖两个维度:摘要的隐含来源及来源披露情况。结果显示,作者在修改人类撰写的摘要时修改量最大,而对未标注来源的AI生成摘要修改更多,常因感知到更高的可读性。一旦披露来源,两类摘要的修改量趋于一致。审稿决策不受摘要来源影响,但与修改数量显著相关。特别是对AI生成摘要,在披露来源的情况下进行细致风格修改能显著提升录用概率。研究发现,经最少修改即可使AI生成摘要达到与人工撰写相当的可接受水平,而对AI作者身份的认知而非客观质量,主导了大部分修改行为。研究强调了在协作式科学写作中源信息披露的重要意义。

原文摘要 · Abstract (English)

Large Language Models have seen expanding application across domains, yet their effectiveness as assistive tools for scientific writing - an endeavor requiring precision, multimodal synthesis, and domain expertise - remains insufficiently understood. We examine the potential of LLMs to support domain experts in scientific writing, with a focus on abstract composition. We design an incentivized randomized controlled trial with a hypothetical conference setup where participants with relevant expertise are split into an author and reviewer pool. Inspired by methods in behavioral science, our novel incentive structure encourages authors to edit the provided abstracts to an acceptable quality for a peer-reviewed submission. Our 2 x 2 between-subject design expands into two dimensions: the implicit source of the provided abstract and the disclosure of it. We find authors make most edits when editing human-written abstracts compared to AI-generated abstracts without source attribution, often guided by higher perceived readability in AI generation. Upon disclosure of source information, the volume of edits converges in both source treatments. Reviewer decisions remain unaffected by the source of the abstract, but bear a significant correlation with the number of edits made. Careful stylistic edits, especially in the case of AI-generated abstracts, in the presence of source information, improve the chance of acceptance. We find that AI-generated abstracts hold potential to reach comparable levels of acceptability to human-written ones with minimal revision, and that perceptions of AI authorship, rather than objective quality, drive much of the observed editing behavior. Our findings reverberate the significance of source disclosure in collaborative scientific writing.

AI写作科学写作源披露评估实验

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。