arXiv:2604.27453cs.CL2026-04ACL

针对写作生成任务,提出细粒度评估与训练方法,提升模型对具体要求的遵循能力。

From Coarse to Fine: Benchmarking and Reward Modeling for Writing-Centric Generation Tasks

论文配图:From Coarse to Fine: Benchmarking and Reward Modeling for Writing-Centric Generation Tasks
图 1 · 摘自论文原文
  • 构建细粒度评估框架WEval,覆盖多任务和多需求类型
  • 在多个写作基准上显著提升性能,通用性更强
  • 适合需要精准满足写作要求的研究者与开发者

大型语言模型在文本生成方面取得显著进展,但在生成式写作任务中仍存在不足。现有评估基准对写作奖励模型的评价过于粗略,无法从具体要求角度衡量性能;现有训练方法要么采用大模型作为评判者,要么训练粗粒度奖励模型,缺乏对细粒度需求遵循的建模。为此,我们提出细粒度评估流程WEval和细粒度强化学习训练框架WRL。WEval的评估数据涵盖多种任务类别和需求类型,通过测量奖励模型排名与黄金排名的相关性,实现系统化评估。WRL通过选择性剔除指令中的需求来构建正负样本,实现更精确的奖励模型训练。实验表明,所提方法在多个写作基准上均取得显著提升,并具备强泛化能力。代码与数据已公开于https://github.com/Rainier-rq1/From_Coarse_to_Fine。

原文摘要 · Abstract (English)

Large language models have achieved remarkable progress in text generation but still struggle with generative writing tasks. In terms of evaluation, existing benchmarks evaluate writing reward models coarsely and fail to measure performance from the perspective of specific requirements. In terms of training, existing training methods either use LLM-as-a-judge approaches or train coarse-grained reward models, lacking fine-grained requirement-adherence reward modeling. To address these issues, we propose a fine-grained evaluation pipeline WEval for writing reward models and a fine-grained reinforcement learning training framework WRL. The evaluation data of WEval covers multiple task categories and requirement types, enabling systematic evaluation of writing reward models by measuring the correlation between the rankings of the reward model and gold rankings. WRL constructs positive and negative samples by selectively dropping instruction requirements, allowing for more precise reward model training. Experiments show that our models achieve substantial improvements across various writing benchmarks and exhibit strong generalization. The code and data are publicly available at \href{https://github.com/Rainier-rq1/From_Coarse_to_Fine}{https://github.com/Rainier-rq1/From\_Coarse\_to\_Fine}.

写作生成奖励建模强化学习评估基准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。