让大模型评作文更准：结合推理过程与数值预测的双阶段训练法
TRACT: Regression-Aware Fine-tuning Meets Chain-of-Thought Reasoning for LLM-as-a-Judge
- 分两阶段训练：先学推理链，再学分数预测
- 在四个数据集上显著优于现有方法，最高提升6.2%准确率
- 适合需要精准文本评分的评测场景，如教育评估
LLM-as-a-judge范式利用大语言模型对文本进行自动化评分，根据评分标准为输入文本分配数值评分。现有方法使用交叉熵损失进行微调，忽略了评分的数值特性。近期工作通过回归感知微调改善数值预测，但未考虑链式思维（CoT）推理。本文提出TRACT（两阶段回归感知微调结合链式思维），将CoT推理与回归感知训练结合。TRACT分为两个阶段：第一阶段微调种子模型生成链式思维，作为第二阶段的监督信号；第二阶段联合使用交叉熵损失学习推理能力、回归感知损失进行分数预测。在四个LLM-as-a-judge数据集和两种LLM上的实验表明，TRACT显著优于现有方法。大量消融实验证明了各组件的重要性。
原文摘要 · Abstract (English)
The LLM-as-a-judge paradigm uses large language models (LLMs) for automated text evaluation, where a numerical assessment is assigned by an LLM to the input text following scoring rubrics. Existing methods for LLM-as-a-judge use cross-entropy (CE) loss for fine-tuning, which neglects the numeric nature of score prediction. Recent work addresses numerical prediction limitations of LLM fine-tuning through regression-aware fine-tuning, which, however, does not consider chain-of-thought (CoT) reasoning for score prediction. In this paper, we introduce TRACT (Two-stage Regression-Aware fine-tuning with CoT), a method combining CoT reasoning with regression-aware training. TRACT consists of two stages: first, seed LLM is fine-tuned to generate CoTs, which serve as supervision for the second stage fine-tuning. The training objective of TRACT combines the CE loss for learning the CoT reasoning capabilities, and the regression-aware loss for the score prediction. Experiments across four LLM-as-a-judge datasets and two LLMs show that TRACT significantly outperforms existing methods. Extensive ablation studies validate the importance of each component in TRACT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。