ADWIN通过自适应窗口机制，让模型训练更省时高效。
ADWIN: Adaptive Windows for Horizon-Aware On-Policy Distillation
- 根据教师反馈动态调整训练窗口长度，优先使用短前缀
- 相比完整轨迹训练，训练成本降低4.1倍，准确率相当或更好
- 适合需要高效训练的数学与代码推理任务
在策略蒸馏（OPD）中，学生模型通过自身生成轨迹接收教师反馈来学习，但传统全轨迹训练会因每次更新都需完整回溯而代价高昂，并在后期位置过度分配监督信号。本文提出ADWIN，一种基于可接受性判断的自适应窗口框架：以教师锚定的短前缀进行训练，用延迟的完整轨迹探针检测前缀与全轨迹对齐情况，并通过过时控制动态调整下一轮训练范围。在单任务、多任务及强弱模型设置下的数学与代码推理基准上，ADWIN显著优化了准确率-计算成本权衡，端到端训练成本最高降低4.1倍，同时保持或超越现有基线性能。
原文摘要 · Abstract (English)
On-policy distillation (OPD) transfers reasoning behavior by training a student on teacher feedback along student-generated trajectories, but standard full-rollout training ties every update to a costly completion and can over-allocate supervision to late positions with low marginal value for the current student. We revisit this assumption through the useful supervision horizon: student-induced rollouts can drift from teacher-preferred continuations, while aligned prefixes may already preserve the long-horizon OPD update direction. We propose ADWIN, an adaptive-window framework for OPD that treats rollout length as an online admissibility decision, training on short teacher-anchored prefixes while using delayed full-rollout probes to audit prefix--full alignment and adapt the next horizon with staleness control. Across math and code reasoning benchmarks in single-task, multi-task, and strong-to-weak settings, ADWIN improves the accuracy--compute trade-off over full-rollout OPD and prefix-based baselines, reducing end-to-end training cost by up to 4.1 times while achieving comparable or better accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。