用大模型模拟新手，生成高质量教学对话数据。
Building Scaffolding Dialogue Data with LLM-Simulated Novices
- 用LLM模拟新手教师，人类专家实时指导，构建教学对话
- 生成对话与真实记录相当，且专家反馈更深入
- 适合教育AI、教学研究者，提升模型教学能力
高质量的多轮教学对话对发展支持教学、学习和决策的AI系统至关重要。这类对话常涉及支架式教学——专家通过提问、反馈和逐步引导支持新手思考。但因隐私顾虑和求助行为的敏感性，真实数据稀缺。本文提出SimInstruct，一种专家在环的可扩展工具，以教学发展辅导为场景，利用大模型模拟不同特质的新手教师（如外向、内向），人类专家提供多轮反馈与指导。该方法无需真实新手参与，即可生成逼真、富有教育意义的对话。结果表明，角色性格显著影响专家互动方式；模拟对话在教学相关性和认知深度上媲美真实录音。专家反馈过程本身也具反思性，提升了数据质量与自身专业洞察。我们基于增强数据微调了LLaMA模型，其教学表现超越GPT-4o，后者存在弱反思提问、过度使用通用表扬、语气居高临下及建议过载等问题。
原文摘要 · Abstract (English)
High-quality, multi-turn instructional dialogues between novices and experts are essential for developing AI systems that support teaching, learning, and decision-making. These dialogues often involve scaffolding -- the process by which an expert supports a novice's thinking through questions, feedback, and step-by-step guidance. However, such data are scarce due to privacy concerns in recording and the vulnerability inherent in help-seeking. We present SimInstruct, a scalable, expert-in-the-loop tool for collecting scaffolding dialogues. Using teaching development coaching as an example domain, SimInstruct simulates novice instructors via LLMs, varying their teaching challenges and LLM's persona traits, while human experts provide multi-turn feedback, reasoning, and instructional support. This design enables the creation of realistic, pedagogically rich dialogues without requiring real novice participants. Our results reveal that persona traits, such as extroversion and introversion, meaningfully influence how experts engage. Compared to real mentoring recordings, SimInstruct dialogues demonstrate comparable pedagogical relevance and cognitive depth. Experts also reported the process as engaging and reflective, improving both data quality and their own professional insight. We further fine-tuned a LLaMA model to be an expert model using the augmented dataset, which outperformed GPT-4o in instructional quality. Our analysis highlights GPT-4o's limitations in weak reflective questioning, overuse of generic praise, a condescending tone, and a tendency to overwhelm novices with excessive suggestions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。