用大模型生成评语，让作文评分更精准透明。
Rationale Behind Essay Scores: Enhancing S-LLM's Multi-Trait Essay Scoring with Rationale Generated by LLMs
- 用大模型按评分维度生成具体评语，指导评分
- 在ASAP等数据集上比现有方法提升显著
- 适合教育评估、AI助教等需要解释的场景
现有自动作文评分系统仅依赖文本内容，未利用评分理由，错失了细粒度捕捉评分标准的机会。本文提出基于理由的多维度作文评分（RMTS），结合提示工程的大模型（LLM）与微调的小型大模型（S-LLM），通过独立的LLM代理根据评分标准生成各维度的特定理由，供评分模型使用以准确预测多维度分数。在ASAP、ASAP++和Feedback Prize等基准数据集上的实验表明，RMTS显著优于当前最优模型和基础S-LLM，在各维度评分上表现更佳。通过引入细粒度的定性理由，RMTS提升了评分的可靠性，并提供部分解释。代码已开源：https://github.com/BBeeChu/RMTS.git。
原文摘要 · Abstract (English)
Existing automated essay scoring (AES) has solely relied on essay text without using explanatory rationales for the scores, thereby forgoing an opportunity to capture the specific aspects evaluated by rubric indicators in a fine-grained manner. This paper introduces Rationale-based Multiple Trait Scoring (RMTS), a novel approach for multi-trait essay scoring that integrates prompt-engineering-based large language models (LLMs) with a fine-tuning-based essay scoring model using a smaller large language model (S-LLM). RMTS uses an LLM-based trait-wise rationale generation system where a separate LLM agent generates trait-specific rationales based on rubric guidelines, which the scoring model uses to accurately predict multi-trait scores. Extensive experiments on benchmark datasets, including ASAP, ASAP++, and Feedback Prize, show that RMTS significantly outperforms state-of-the-art models and vanilla S-LLMs in trait-specific scoring. By assisting quantitative assessment with fine-grained qualitative rationales, RMTS enhances the trait-wise reliability, providing partial explanations about essays. The code is available at https://github.com/BBeeChu/RMTS.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。