ArguLens实现可解释的作文评分与标签反馈，支持本地部署和开源使用。
ArguLens: An Open-Source System for Automated Essay Scoring and Label-Aware Feedback Generation
- 分解评分流程为三组件：论点识别、特征评分、反馈生成，模块化设计
- 在PERSUADE 2.0上达82.6%分类准确率，加标注后评分提升0.055 QWK
- 支持本地运行、批量处理，适合教育研究与隐私敏感场景
现有自动作文评分系统通常仅输出单一分数，缺乏可解释性证据，且依赖封闭API带来数据隐私与成本问题。我们提出ArguLens，一个开源、可本地部署的系统，将评分任务解耦为三个独立组件：基于PERSUADE 2.0微调的Qwen2.5-7B-Instruct（LoRA）论点分类器、基于31个语言与论述特征的无年级轻量级梯度提升机评分器，以及基于Qwen2.5-14B-Instruct的标签感知反馈生成器（通过vLLM服务）。配套Gradio网页界面支持单篇与批量评分，可下载每篇作文的分解结果。在作文不重叠的PERSUADE 2.0测试集上，分类器准确率达82.6%，宏平均F1为0.727；在提示分组5折交叉验证下，评分器均值QWK达0.813（基于理想论点特征协议），消融实验表明加入真实论点标注使评分提升+0.055 QWK（配对t检验，p=0.010）。该结果为组件级诊断，非端到端模型表现。反馈生成器附带结构化评估协议，人工评价留待后续工作。系统已按Apache 2.0协议开源，地址：https://github.com/wwrwbs/AI_AWE。
原文摘要 · Abstract (English)
Most automated essay scoring (AES) systems output a single holistic score without interpretable evidence and rely on closed APIs that introduce data privacy and cost barriers. We present ArguLens, an opensource, locally deployable system that decomposes AES into three decoupled components: a discourse-move classifier (Qwen2.5-7B-Instruct fine-tuned with LoRA on PERSUADE 2.0), a grade-independent LightGBM scorer over 31 linguistic and discourse features, and a label-aware feedback generator served through vLLM with a Qwen2.5-14BInstruct backbone. A Gradio web UI exposes pluggable inference backends and supports single-essay and batch scoring with downloadable per-essay breakdowns. On an essaydisjoint PERSUADE 2.0 test split, the logitprobe classifier achieves 82.6% accuracy and 0.727 macro-F1; under prompt-grouped 5-fold cross-validation the scorer reaches a mean QWK of 0.813 under an oracle discoursefeature protocol, and an ablation shows that adding gold discourse annotations yields an increment of +0.055 QWK over the lexical+syntactic configuration (paired t-test, p = 0.010). This is a component-level diagnostic rather than an end-to-end classifier-to-scorer result. The feedback generator ships with a structured evaluation protocol; its human-rater study is left to future work. The system is released under Apache 2.0 at https://github.com/wwrwbs/AI_AWE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。