提出可量化文本中AI修改程度的新方法，区分人类与AI混合写作。
EditLens: Quantifying the Extent of AI Editing in Text
- 用轻量级相似性度量评估文本被AI修改的程度。
- 在二分类和三分类任务中准确率分别达94.7%和90.4%。
- 适用于学术诚信、教育评估及写作工具影响分析。
大量大语言模型查询请求是基于用户原始文本进行编辑，而非从零生成。现有研究主要关注完全由AI生成文本的检测，而本文首次证明AI编辑过的文本可与人类写作及纯AI生成文本区分开来。我们提出使用轻量级相似性度量，结合人工标注验证其有效性，并以此作为中间监督信号，训练出名为EditLens的回归模型，用于预测文本中AI编辑的程度。该模型在二分类（F1=94.7%）和三分类（F1=90.4%）任务中均达到当前最优性能。研究不仅证明了AI编辑文本可被检测，还实现了对修改程度的量化，对作者归属、教育监管与政策制定具有重要意义。作为案例研究，我们分析了Grammarly等主流写作辅助工具的AI编辑效果。为推动后续研究，团队承诺公开发布模型与数据集。
原文摘要 · Abstract (English)
A significant proportion of queries to large language models ask them to edit user-provided text, rather than generate new text from scratch. While previous work focuses on detecting fully AI-generated text, we demonstrate that AI-edited text is distinguishable from human-written and AI-generated text. First, we propose using lightweight similarity metrics to quantify the magnitude of AI editing present in a text given the original human-written text and validate these metrics with human annotators. Using these similarity metrics as intermediate supervision, we then train EditLens, a regression model that predicts the amount of AI editing present within a text. Our model achieves state-of-the-art performance on both binary (F1=94.7%) and ternary (F1=90.4%) classification tasks in distinguishing human, AI, and mixed writing. Not only do we show that AI-edited text can be detected, but also that the degree of change made by AI to human writing can be detected, which has implications for authorship attribution, education, and policy. Finally, as a case study, we use our model to analyze the effects of AI-edits applied by Grammarly, a popular writing assistance tool. To encourage further research, we commit to publicly releasing our models and dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。