用强化学习动态选提示,让大模型作文评分更省成本且精准
Learning to Grade Efficiently: A Bandit-Driven Prompt-Selection Framework for Low-Cost LLM Essay Scoring

- 把不同提示当作老虎机的杆子,实时选择最优评分策略
- 减少78.4%的模型调用次数,准确率接近全量搜索结果
- 适合关注评分成本与效果平衡的教育科技平台使用
大型语言模型在自动作文评分(AES)中表现强劲,但现有方法多采用固定提示,无法应对运行成本问题和提示配置的动态优化需求。本文提出一种新型成本感知方法,将每种提示类型视为多臂老虎机(MAB)控制器中的一个臂,实现在推理过程中自适应选择最优提示策略。我们在雅思写作任务2的作文上进行实验,结果表明,该MAB框架在达到与穷举网格搜索相当的评分准确率的同时,将大模型调用次数减少了78.4%,成功找到最佳评分方案。我们实现了四种不同的评分配方(多步与单步评估、带与不带校准示例),发现带示例的多步评估方式准确率最高。通过追踪分词消耗与延迟,并结合一致性指标,首次生成了作文评分的成本-可靠性学习曲线,为教育科技平台在运营成本与评估有效性之间提供可操作的决策支持。本工作首次将在线控制机制应用于AES中的提示策略自适应选择,将提示选择从离线超参数优化问题转变为高效在线学习任务。
原文摘要 · Abstract (English)
Large Language Models (LLMs) demonstrate strong capabilities in automated essay scoring (AES), but contemporary approaches typically employ fixed prompt selection, failing to address operational cost concerns and evolving optimal configurations. We propose a novel cost-aware approach that treats each prompt type as an arm in a multi-armed bandit (MAB) controller, enabling adaptive selection of optimal prompting strategies during inference. Our experiments on IELTS Writing Task 2 essays show that the MAB framework achieves comparable scoring accuracy to exhaustive grid search while reducing LLM calls by 78.4\% to find the best grading approach. We implemented four distinct grading recipes (multi-step vs. single-step assessment, with vs. without calibration examples) and found that the multi-step approach with examples achieves the highest accuracy. By tracking token usage and latency alongside agreement metrics, we produce the first cost-reliability learning curves for essay scoring, providing actionable insights for educational technology platforms that must balance operational costs against assessment validity. This work represents the first application of online control mechanisms to adaptively select prompting strategies in AES, transforming prompt selection from an offline hyperparameter optimization problem into an efficient online learning task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。