arXiv:2605.13136cs.CL2026-05

通过动态门控教师模型,提升小模型的逻辑推理稳定性。

GateKD: Confidence-Gated Closed-Loop Distillation for Robust Reasoning

论文配图:GateKD: Confidence-Gated Closed-Loop Distillation for Robust Reasoning
图 1 · 摘自论文原文
  • 用教师置信度控制知识传递,形成闭环反馈
  • 在逻辑与符号推理上显著优于传统方法
  • 适合构建高可靠性的轻量级推理模型

将大语言模型的多步推理能力蒸馏到小型学生模型仍面临噪声推理、幻觉监督和静态师生交互的挑战。现有方法多为开环模式,隐含假设教师始终可靠,导致错误推理传播。本文提出GateKD,一种基于置信度门控的闭环蒸馏框架,将教师视为动态把关者而非静态权威。其包含三个互补机制:(i) 置信度门控的软监督,仅蒸馏可信预测信号;(ii) 门控隐藏状态演化,仅在教师置信度高时对齐中间表示;(iii) 可靠性过滤注意力蒸馏,保留稳定推理结构并抑制噪声模式。三者共同构成闭环反馈,使教师置信度持续调节蒸馏过程,减少幻觉传递并稳定学生推理。在共情、逻辑与符号推理基准上,使用T5与Flan-T5不同规模模型进行实验,结果表明GateKD持续优于强开环基线。尤其在逻辑与符号推理任务中表现突出,在低资源设置下仍具鲁棒性,且任一门控组件缺失均导致性能明显下降。结果表明,置信度门控闭环监督对构建可靠可扩展的小型推理模型至关重要。

原文摘要 · Abstract (English)

Distilling multi-step reasoning abilities from large language models (LLMs) into compact student models remains challenging due to noisy rationales, hallucinated supervision, and static teacher-student interactions. Existing reasoning distillation methods, including mentor-based approaches, predominantly operate in an open-loop manner, implicitly assuming uniform teacher reliability and consequently propagating erroneous intermediate reasoning. We propose GateKD, a confidence-gated closed-loop distillation framework that enables robust reasoning transfer by treating the teacher as a dynamic gatekeeper rather than a static oracle. GateKD introduces three complementary mechanisms: (i) confidence-gated soft supervision that selectively distills reliable predictive signals, (ii) gated hidden-state evolution that aligns intermediate representations only when teacher confidence is high, and (iii) reliability-filtered attention distillation that preserves stable reasoning structures while suppressing noisy patterns. These components jointly form a closed feedback loop in which teacher confidence continuously modulates the distillation process, reducing hallucination transfer and stabilizing student reasoning. Extensive experiments across commonsense, logical, and symbolic reasoning benchmarks, using T5 and Flan-T5 backbones of varying sizes, demonstrate that GateKD consistently outperforms strong open-loop distillation baselines. Notably, GateKD yields substantial gains in logical and symbolic reasoning, remains robust under low-resource distillation settings, and shows clear performance degradation when any gating component is removed. Our results highlight that confidence-gated closed-loop supervision is critical for building reliable and scalable small reasoning models.

模型蒸馏推理增强闭环学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。