用双模型生成精准评语,提升自动作文评分的可解释性与准确性。
Two Heads Are Better Than One: Dual-Model Verbal Reflection at Inference-Time
- 通过对比推理图路径差异,合成精确的口语化反馈数据。
- DARS框架在多个评估指标上超越现有基线,表现更稳定。
- 适合需要可解释评分的教育AI场景,如自动作文打分。
尽管偏好优化方法提升了大语言模型的推理能力,但往往无法透明解释为何一种推理结果优于另一种。这一局限在自动学生作答评分(ASAS)中尤为关键,因为可解释性对评估结果的合理性至关重要。口头强化学习虽有生成显式反思的潜力,却常产生浅层批评,反而损害评分性能。现有大模型也难以可靠识别ASAS任务中的细微推理错误。此外,人工标注中间推理错误成本高且难扩展。为此,我们提出一种对比式反思合成流程,通过识别推理图路径的差异生成精准口语反馈。基于这些合成反思数据,我们设计了双模型反思评分框架DARS,包含一个专门训练用于有效反思的评论模型。DARS在各项评估指标上均表现出色,持续优于现有ASAS基线。大量实验进一步揭示了反思数据的价值、框架设计及DARS的缩放行为。代码已开源:https://github.com/lijiazheng99/DARS。
原文摘要 · Abstract (English)
Although preference optimization methods have improved reasoning performance in Large Language Models (LLMs), they often lack transparency regarding why one reasoning outcome is preferred over another. This limitation is especially critical in Automated Student Answer Scoring (ASAS), where explainability is essential to justify assessment outcomes. Verbal reinforcement learning offers the potential to generate explicit reflection, but it tends to produce superficial critiques that can harm assessment performance. Existing LLMs also struggle to reliably detect subtle reasoning errors in ASAS tasks. Moreover, manually identifying intermediate reasoning errors is expensive and difficult to scale. To address these challenges, we introduce a contrastive reflection synthesis pipeline that generates precise verbal feedback by identifying discrepancies in structure reasoning graph paths. Leveraging these synthetic reflection data, we propose DARS, a Dual-model Reflective Scoring framework featuring a dedicated Critic model trained for effective reflection. DARS achieves strong performance and consistently outperforms existing ASAS baselines across all evaluation metrics. Extensive experiments further provide novel insights into the value of reflection data, framework design, and the scaling behavior of DARS. We release the DARS code at https://github.com/lijiazheng99/DARS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。