解决大模型作文评分因长度限制导致评估不全的问题
Long Context Automated Essay Scoring with Language Models
- 采用改进版Transformer架构处理超长作文输入
- 在Kaggle ASAP 2.0数据集上实现更完整的评分表现
- 适合需要长文本评估的教育自动化场景
基于Transformer的语言模型在架构上受限于固定最大输入长度。高年级学生撰写的作文常超出多数开源模型的长度上限。当前常用方法是截断输入，但这严重削弱了模型对评分标准中组织结构等要素的完整捕捉能力。本研究在Kaggle ASAP 2.0数据集上，评估了多种通过架构改进突破长度限制的模型，包括微调后的XLNet、Longformer、ModernBERT、Mamba及Llama模型。
原文摘要 · Abstract (English)
Transformer-based language models are architecturally constrained to process text of a fixed maximum length. Essays written by higher-grade students frequently exceed the maximum allowed length for many popular open-source models. A common approach to addressing this issue when using these models for Automated Essay Scoring is to truncate the input text. This raises serious validity concerns as it undermines the model's ability to fully capture and evaluate organizational elements of the scoring rubric, which requires long contexts to assess. In this study, we evaluate several models that incorporate architectural modifications of the standard transformer architecture to overcome these length limitations using the Kaggle ASAP 2.0 dataset. The models considered in this study include fine-tuned versions of XLNet, Longformer, ModernBERT, Mamba, and Llama models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。