用分拆架构让AI tutor更可控、可解释，避免过早给答案。
From Untamed Black Box to Interpretable Pedagogical Orchestration: The Ensemble of Specialized LLMs Architecture for Adaptive Tutoring
- 分角色协作：规则引擎指挥多个专用AI，分工负责教学、评估等任务。
- 严格遵守教学规则，提示使用效率提升3.3倍，100%执行‘先尝试再提示’。
- 适合教育AI研发者，尤其关注可解释性与成本控制的团队。
教育对话中常用的单体大语言模型常如黑箱，难以审计且常提前给出答案。本文提出分立式专用LLM集成架构（ES-LLMS），将决策与表达分离：由基于规则的协调器调度覆盖教学、评估、反馈等职能的专用代理，并受可解释的贝叶斯知识追踪（BKT）模型指导。一个LLM渲染器负责生成自然语言输出。该设计强化可靠性与可控性：如“先尝试后提示”等教学约束以显式规则强制执行，系统记录每轮动作轨迹与约束检查结果。人工专家评审（N=6）与多大模型评委组（六款SOTA模型）验证显示，ES-LLMS在91.7%和79.2%案例中更受青睐。其在七个维度均显著优于单体基线，尤其在支架引导与信任可解释性上。蒙特卡洛模拟（N=2,400）揭示了‘掌握增益悖论’——单体模型因过度协助虚增短期表现。而ES-LLMS实现100%教学规则遵循，提示效率提升3.3倍。运营层面，通过无状态提示降低54%成本与22%延迟。结论：结构解耦是构建可信、可验证、高效教育智能体的关键。
原文摘要 · Abstract (English)
Monolithic Large Language Models (LLMs) used in educational dialogue often behave as "black boxes," where pedagogical decisions are implicit and difficult to audit, frequently violating instructional constraints by providing answers too early. We introduce the Ensemble of Specialized LLMS (ES-LLMS) architecture that separates decision-making from wording. Pedagogical actions are selected by a deterministic rules-based orchestrator coordinating specialized agents covering tutoring, assessment, feedback, scaffolding, motivation and ethics-guided by an interpretable Bayesian Knowledge Tracing (BKT) student model. An LLM renderer surface-realizes the chosen action in natural language. This design emphasizes reliability and controllability: constraints such as "attempt-before-hint" and hint caps are enforced as explicit rules, and the system logs per-turn agent traces and constraint checks. Validation of pedagogical quality via human expert reviewers (N=6) and a multi-LLM-as-Judge panel (six state-of-the-art models) showed that ES-LLMs were preferred in 91.7% and 79.2% of cases, respectively. The architecture significantly outperformed monolithic baselines across all seven dimensions, particularly in Scaffolding & Guidance, and Trust & Explainability. Furthermore, a Monte Carlo simulation (N=2,400) exposed a "Mastery Gain Paradox," where monolithic tutors inflated short-term performance through over-assistance. In contrast, ES-LLMs achieved 100% adherence to pedagogical constraints (e.g., attempt-before-hint) and a 3.3x increase in hint efficiency. Operationally, ES-LLMs reduced costs by 54% and latency by 22% by utilizing stateless prompts. We conclude that structural decoupling is essential for transforming stochastic models into trustworthy, verifiable and resource-efficient pedagogical agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。