强模型在推理时构建代码工具,让弱模型不改参数也能大幅提效。
AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses

- 强模型用少量数据迭代生成推理工具,帮助弱模型完成任务。
- 弱模型性能从0.49提升至0.91,接近翻倍。
- 适合想提升小模型表现又不想重训练的开发者使用。
近期的模型蒸馏工作通常通过教师强制、同策略蒸馏等训练时方法,将大模型能力迁移至小模型。本文探讨能否在推理时实现这种迁移。我们研究强模型构建推理时工具(harness)以增强弱目标模型的表现:强模型利用5%的数据作为验证集,多轮迭代优化其工具,最终在全测试集上评估。实验表明,该方法极为有效,平均性能从0.49提升至0.91。分析显示,性能提升主要源于将不稳定的模型推理转为确定性代码、特定基准路由和严格答案格式约束,而非鼓励目标模型更深入推理或更广泛采样。我们进一步发现,强模型的推理投入与工具质量呈单调提升关系,平台差异影响较小,而弱目标模型获益最大。结果表明,推理时工具设计是传统训练时蒸馏的重要补充,使强模型无需重训练即可传递认知结构给弱模型。
原文摘要 · Abstract (English)
Recent work on distillation transfers the capabilities of large models to smaller ones often by updating the latter's parameters, through teacher forcing, on-policy distillation, and related training-time methods. In this paper, we ask whether such transfer can instead occur at test time. We study strong-to-weak scaffolding: whether a stronger builder model can construct inference-time harnesses that help a weaker target model solve tasks more reliably without any parameter updates. Using four representative Theory-of-Mind benchmarks, each builder model uses 5% of the data as a validation set to iteratively refine its harness over multiple rounds, after which the finalized harness is evaluated on the full test set. Empirically, this form of test-time capability transfer is highly effective, nearly doubling average target-model performance from 0.49 to 0.91. Our analysis shows that the gains come primarily from offloading unstable model reasoning into deterministic code, benchmark-specific routing, and strict answer-format enforcement, rather than from encouraging the target model to reason more extensively or sample more broadly. We further find that builder-model reasoning effort improves harness quality monotonically, platform effects are modest relative to the builder model's own capability, and weaker target models receive the largest gains. These results suggest that inference-time harness design is an important complement to conventional training-time distillation, enabling strong models to transfer cognitive structure to weaker models without retraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。