让大模型学会像人一样精准修改不当论证，保持原意同时提升表达得体性。
Teaching LLMs Human-Like Editing of Inappropriate Argumentation via Reinforcement Learning
- 用强化学习训练模型生成独立可选的句子级修改建议，模仿人类连贯修改习惯。
- 多轮编辑后得体性接近全文重写，自动与人工评估均优于现有方法。
- 适合需要精准、自然文本修正的场景，如学术写作、公共讨论等。
大语言模型在文本编辑中的应用已成常态，例如使论点更适用于讨论。然而，对比人类与模型生成的修改发现：模型常进行零散的多处改动且显著改变原意，而人类则倾向于将相关修改封装为自洽、保意的整块调整。本文提出一种基于强化学习的方法，使大模型学会类人编辑策略，以提升论证得体性。该方法生成可独立接受或拒绝的句级修改建议，并通过组相对策略优化与多组件奖励函数进行训练，联合优化编辑层面的语义相似度、流畅性与模式一致性，以及论点层面的得体性。在自动与人工评估中，该方法在类人编辑方面超越竞争基线及当前最优方案，多轮编辑后的得体性接近全文重写水平。
原文摘要 · Abstract (English)
Editing human-written text has become a standard use case of large language models (LLMs), for example, to make one's arguments more appropriate for a discussion. Comparing human to LLM-generated edits, however, we observe a mismatch in editing strategies: While LLMs often perform multiple scattered edits and tend to change meaning notably, humans rather encapsulate dependent changes in self-contained, meaning-preserving edits. In this paper, we present a reinforcement learning approach that teaches LLMs human-like editing to improve the appropriateness of arguments. Our approach produces self-contained sentence-level edit suggestions that can be accepted or rejected independently. We train the approach using group relative policy optimization with a multi-component reward function that jointly optimizes edit-level semantic similarity, fluency, and pattern conformity as well as argument-level appropriateness. In automatic and human evaluation, it outperforms competitive baselines and the state of the art in human-like editing, with multi-round editing achieving appropriateness close to full rewriting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。