用强化学习统一优化作文评分与反馈生成，提升质量与一致性。
Beyond Score Prediction: LLM-Based Essay Scoring and Feedback Generation via Reinforcement Learning with Rubric Rewards
- 基于评分标准构建细粒度反馈评估框架，支持可量化的反馈训练。
- 在ASAP数据集上实现QWK=0.803的评分性能，优于现有大模型方法。
- 适用于需要高质量自动评分与反馈的教育AI场景，如智能辅导系统。
大型语言模型（LLMs）已广泛用于自动作文评分（AES）和自动反馈生成（AFG）。然而，现有研究主要依赖提示工程或监督微调，对强化学习（RL）后训练及反馈质量自动化评估的系统性研究仍有限。本文提出RLAES，一个通过强化学习联合优化作文评分与反馈生成的统一框架。为使反馈质量可测量、可解释且可用于训练，我们引入基于评分标准的反馈评估（RFE），包含166个细粒度二元评分项及以LLM为裁判的评估机制。在此基础上，提出自适应门控反馈优化（AGFO），按需激活基于评分标准的奖励，降低评估开销并提升反馈质量。同时提出邻近对比推理（ACR），通过显式对比相邻分数等级，改进分数排序校准。实验表明，RFE框架能捕捉作文与反馈的一致性，具备强配对区分能力，且与专家偏好高度一致。在ASAP基准上，RLAES-AGFO在所有基于LLM的方法中取得最佳评分性能（QWK=0.803），同时保持与GPT-5.5相当的反馈质量，并避免了仅基于评分的强化学习带来的反馈退化问题。代码与数据集已公开于https://github.com/hellomuyi/RLAES。
原文摘要 · Abstract (English)
Large language models (LLMs) have been widely applied to automated essay scoring (AES) and automated feedback generation (AFG). However, existing studies rely primarily on prompt engineering or supervised fine-tuning, while systematic research on reinforcement learning (RL) post-training and automated evaluation of feedback quality remains limited. We propose RLAES, a unified LLM framework that jointly optimizes essay scoring and feedback generation through RL. To make feedback quality measurable, interpretable, and usable for training, we introduce Rubric-based Feedback Evaluation (RFE), an essay-grounded feedback evaluation framework comprising 166 fine-grained binary rubric items and an LLM-as-judge. Building on RFE, we propose Adaptive Gated Feedback Optimization (AGFO), which activates rubric-based feedback rewards on demand during RL, reducing evaluation overhead while improving feedback quality. We also propose Adjacent Contrastive Reasoning (ACR) to improve ordinal score calibration by explicitly contrasting adjacent score levels. Experimental results show that the RFE framework captures essay-feedback consistency, exhibits strong pairwise discriminative power, and closely aligns with expert preferences. On the ASAP benchmark, RLAES-AGFO achieves the best scoring performance among LLM-based methods (QWK = 0.803), while maintaining feedback quality comparable to GPT-5.5 and avoiding the feedback degradation observed under score-only RL. Code and datasets are publicly available at https://github.com/hellomuyi/RLAES.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。