对比四种大模型作文评分方法，发现融合微调与检索增强效果最佳。
From Prompting to Preference Optimization: A Comparative Study of LLM-based Automated Essay Scoring
- 采用统一基准测试四种大模型评分方法
- 最优组合k-SFT+RAG达93%准确率
- 适合教育AI开发者与语言评估研究者
大语言模型（LLMs）正重塑自动作文评分（AES），但以往研究多孤立评估单一技术，难以全面理解其在英语作为第二语言（L2）写作中的优劣。为此，本文在雅思写作任务2（IELTS Writing Task 2）上开展系统性比较，评估四种主流方法：(i) 编码器分类微调，(ii) 零样本与少样本提示，(iii) 指令微调结合检索增强生成（RAG），(iv) 监督微调结合直接偏好优化（DPO）与RAG。结果揭示各方法在准确率-成本-鲁棒性间的权衡，最优配置（k-SFT + RAG）实现最高综合性能，F1得分达93%。本研究首次对现代大模型驱动的英语L2作文评分策略进行统一实证比较，为自动评分应用提供重要参考。代码已公开于 https://github.com/MinhNguyenDS/LLM_AES-EnL2。
原文摘要 · Abstract (English)
Large language models (LLMs) have recently reshaped Automated Essay Scoring (AES), yet prior studies typically examine individual techniques in isolation, limiting understanding of their relative merits for English as a Second Language (L2) writing. To bridge this gap, we presents a comprehensive comparison of major LLM-based AES paradigms on IELTS Writing Task~2. On this unified benchmark, we evaluate four approaches: (i) encoder-based classification fine-tuning, (ii) zero- and few-shot prompting, (iii) instruction tuning and Retrieval-Augmented Generation (RAG), and (iv) Supervised Fine-Tuning combined with Direct Preference Optimization (DPO) and RAG. Our results reveal clear accuracy-cost-robustness trade-offs across methods, the best configuration, integrating k-SFT and RAG, achieves the strongest overall results with F1-Score 93%. This study offers the first unified empirical comparison of modern LLM-based AES strategies for English L2, promising potential in auto-grading writing tasks. Code is public at https://github.com/MinhNguyenDS/LLM_AES-EnL2
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。