通过表征对齐防止大模型导师在问答中提前暴露答案
Mitigating Scaffolding Collapse in Socratic Tutors via Representation Alignment
- 先微调再用偏好优化+表征损失，保持对话中引导状态分离
- 在5个学科5种攻击下，崩溃率降至32%，平均9轮后才崩溃
- 适合需要长期引导式教学的AI导师研发与评估
基于大语言模型的苏格拉底式导师在多轮问答中引导学生，但可能因持续压力导致支架坍塌：导师逐渐放弃引导提问，直接给出答案。现有防御方法主要通过提示、偏好优化或过滤来约束输出，未能解决导致崩溃的内部表征漂移。本文提出支架保留表征对齐（Scaffold-Preserving Representation Alignment）框架，分两阶段进行：先用监督微调初始化导师，再结合轨迹加权直接偏好优化与锚定冻结参考状态的边界保持表征损失。该方法旨在维持对话各轮中支架保留与坍塌诱导隐藏状态的分离。我们在五个STEM领域和五种红队攻击策略下评估该方法。在Qwen3-8B上，崩溃率降至32%，平均崩溃时间延至九轮以上，同时过拒率保持较低，表明表征级对齐可显著提升长程苏格拉底式辅导的鲁棒性。
原文摘要 · Abstract (English)
Large language model (LLM)-based Socratic tutors increasingly guide students through multi-turn questioning, but they can suffer from scaffolding collapse: under sustained student pressure, a tutor gradually abandons guided inquiry and reveals solutions directly. Prior defenses primarily constrain observable responses through prompting, preference optimization, or filtering, leaving the internal representation drift that precedes trajectory-level collapse largely unaddressed. We propose Scaffold-Preserving Representation Alignment, a two-stage framework that first warms up a Socratic tutor with supervised fine-tuning, then combines trajectory-weighted direct preference optimization with a margin-preserving representation loss anchored to frozen reference states. Our method is designed to maintain separation between scaffold-preserving and collapse-inducing hidden states across dialogue turns. We evaluate our method across five STEM disciplines and five red-teaming attack strategies. On Qwen3-8B, our method lowers Collapse Rate to 32%, delays average collapse onset beyond nine turns, and keeps over-refusal low, suggesting that representation-level alignment can improve the robustness of long-horizon Socratic tutoring under our red-teaming protocol.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。