用小模型实现作文论点识别与评估,兼顾效率与隐私。
Leveraging Small LLMs for Argument Mining in Education: Argument Component Identification, Classification, and Assessment
- 用小规模开源大模型通过提示和微调完成论点拆分、分类与质量评估。
- 在6-12年级作文数据集上,微调后模型在论点分割和分类上优于基线。
- 适合教育场景本地部署,实时反馈且保护学生隐私。
论点挖掘算法可分析作文的论证结构,为教育提供针对性反馈,提升学生论证能力。现有方法多采用编码器或编码器-解码器架构,而解码器仅模型尚未被充分探索。本文提出利用开源小型大语言模型(LLM)通过少样本提示和微调实现论点挖掘。其小体积与开源特性保障了易用性、隐私性和计算效率,支持学校本地部署。具体任务包括:将学生作文切分为论点单元、按类型分类论点、评估论点质量。在针对6-12年级学生的Feedback Prize - Predicting Effective Arguments数据集上进行实验,结果表明微调后的小模型在论点分割与类型识别上优于基线方法,而少样本提示在质量评估上表现相当。该研究凸显了小型开源模型在提供低延迟、个性化反馈方面的教育潜力,有助于自主学习与写作能力提升,同时保证低成本与高隐私性。
原文摘要 · Abstract (English)
Argument mining algorithms analyze the argumentative structure of essays, making them a valuable tool for enhancing education by providing targeted feedback on the students' argumentation skills. While current methods often use encoder or encoder-decoder deep learning architectures, decoder-only models remain largely unexplored, offering a promising research direction. This paper proposes leveraging open-source, small Large Language Models (LLMs) for argument mining through few-shot prompting and fine-tuning. These models' small size and open-source nature ensure accessibility, privacy, and computational efficiency, enabling schools and educators to adopt and deploy them locally. Specifically, we perform three tasks: segmentation of student essays into arguments, classification of the arguments by type, and assessment of their quality. We empirically evaluate the models on the Feedback Prize - Predicting Effective Arguments dataset of grade 6-12 students essays and demonstrate how fine-tuned small LLMs outperform baseline methods in segmenting the essays and determining the argument types while few-shot prompting yields comparable performance to that of the baselines in assessing quality. This work highlights the educational potential of small, open-source LLMs to provide real-time, personalized feedback, enhancing independent learning and writing skills while ensuring low computational cost and privacy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。