arXiv:2606.10327cs.CLcs.LG2026-06

按论述结构顺序微调大模型,让自动作文评分更准确

The Order Matters: Sequential Fine-Tuning of LLaMA for Coherent Automated Essay Scoring

论文配图:The Order Matters: Sequential Fine-Tuning of LLaMA for Coherent Automated Essay Scoring
图 1 · 摘自论文原文
  • 按引言、立场、论点等顺序逐步微调模型,模拟人类写作逻辑
  • 结论评分准确率达85%,证据评分F1达65%,优于独立训练
  • 小模型胜过超大模型,适合资源有限的教育AI应用

自动作文评分系统需判断引言、立场、论点、证据、结论等相互关联的论述元素,但现有方法多孤立处理,影响连贯性与泛化能力。本文针对LLaMA-3.1-8B进行任务感知微调,采用4比特量化与参数高效LoRA技术,对比三种训练方案:(i)顺序微调(依次在引言、立场、论点、证据、结论上训练),(ii)独立模型(各任务单独建模),(iii)随机多任务。在PERSUADE~2.0数据集上的实验表明:顺序微调效果最佳,结论准确率达85%(对应F1为87%),证据F1达65%(准确率63%),优于独立训练,且在结论评分上超越容量更大的通用版LLaMA-70B。随机训练虽提升立场评分(F1 57%),但整体不一致。结果说明:与论述结构对齐的训练课程可显著提升评分性能;经过优化的小模型可媲美大型语言模型,为低成本、可扩展的教育评估提供可行路径。代码与模板已公开。

原文摘要 · Abstract (English)

Automated Essay Scoring (AES) systems must judge interdependent discourse elements (e.g., lead, claim, evidence, conclusion), yet most approaches treat these in isolation, harming coherence and generalization. We investigate task-aware fine-tuning of LLaMA-3.1-8B for AES using parameter-efficient LoRA with 4-bit quantization and compare three training curricula: (i) Sequential (progressively fine-tuning on lead, then position, then claim, then evidence, then conclusion), (ii) Independent (task-specific models), and (iii) Randomized (shuffled multi-task). Experiments on the PERSUADE~2.0 corpus show that modeling task dependencies matters: Sequential fine-tuning yields the strongest overall results, including F1 scores of 65% (evidence) and 87% (conclusion) and corresponding accuracies of 63% and 85%, surpassing Independent training and outperforming a general-purpose LLaMA-70B baseline on conclusion despite its far larger capacity. Randomized training improves position scoring (57% F1) but is less consistent elsewhere. These findings indicate that (1) curriculum design aligned with discourse structure can materially improve AES, and (2) small, task-optimized models can be competitive with substantially larger Large Language Models (LLM), offering a practical path to scalable, cost-effective assessment. We release templates and implementation details to facilitate reproduction and future work on curriculum design for educational NLP.

自动评分大模型微调教育AILoRA

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。