用检索增强提示提升AI导师纠错能力，效果优于传统方法。
NeuralNexus at BEA 2025 Shared Task: Retrieval-Augmented Prompting for Mistake Identification in AI Tutors
- 通过检索相似案例构建结构化提示，结合LLM推理
- 在BEA 2025任务中取得最优性能，超越所有基线
- 适合需要可解释性教学反馈的AI教育系统开发者
本文介绍我们在BEA 2025共享任务第1赛道——评估AI导师教学能力中的系统。该任务要求判断导师回复是否正确识别了学生数学推理中的错误。我们尝试了四种方法：（1）基于多个预训练语言模型池化嵌入的集成学习模型；（2）使用[CLS]嵌入的冻结Sentence-Transformer配合MLP分类器；（3）融合历史与回复嵌入的多头注意力机制模型；（4）基于GPT-4o的检索增强少样本提示系统。最终系统通过检索语义相似示例、构建结构化提示，并采用模式引导输出解析，实现可解释预测。实验表明，该方法显著优于所有基线，验证了示例驱动提示与大模型推理结合在教学反馈评估中的有效性。代码已开源：https://github.com/NaumanNaeem/BEA_2025。
原文摘要 · Abstract (English)
This paper presents our system for Track 1: Mistake Identification in the BEA 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutors. The task involves evaluating whether a tutor's response correctly identifies a mistake in a student's mathematical reasoning. We explore four approaches: (1) an ensemble of machine learning models over pooled token embeddings from multiple pretrained language models (LMs); (2) a frozen sentence-transformer using [CLS] embeddings with an MLP classifier; (3) a history-aware model with multi-head attention between token-level history and response embeddings; and (4) a retrieval-augmented few-shot prompting system with a large language model (LLM) i.e. GPT 4o. Our final system retrieves semantically similar examples, constructs structured prompts, and uses schema-guided output parsing to produce interpretable predictions. It outperforms all baselines, demonstrating the effectiveness of combining example-driven prompting with LLM reasoning for pedagogical feedback assessment. Our code is available at https://github.com/NaumanNaeem/BEA_2025.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。