KAT-V1让大模型自动判断何时思考,提升推理效率与准确性。
KAT-V1: Kwai-AutoThink Technical Report
- 通过动态切换推理与非推理模式,根据任务复杂度智能决策。
- 在多个基准测试中超越同类开源模型,减少30%以上token消耗。
- 适合需要高效精准推理的工业级应用,如代码生成与智能助手。
我们提出Kwaipilot-AutoThink(KAT),一个400亿参数的开源大语言模型,旨在解决推理密集型任务中的过度思考问题。通过一种新型自动思维训练范式,系统可根据任务复杂度动态切换推理与非推理模式。首先,基于创新的标注流程和多智能体合成策略构建双模式数据集;随后采用增强型多标记预测知识蒸馏,实现高效细粒度推理迁移,且预训练成本极低。此外,引入冷启动初始化策略,利用多数投票信号和意图感知提示建立模式选择先验。最后,提出Step-SRPO强化学习算法,在GRPO框架中融入中间监督,对推理模式选择与回答准确率提供结构化引导。大量实验表明,KAT在多个基准上持续达到或超越当前最优模型(如DeepSeek-R1-0528和Qwen3-235B-A22B),涵盖广泛推理任务,同时显著降低令牌使用量。特别地,在受控泄露的LiveCodeBench Pro上,其表现超越所有开源模型,甚至优于o3-mini。除学术评估外,KAT已成功部署于快手内部编码助手Kwaipilot,显著提升真实开发流程的准确性、效率与可控制推理行为。目前,我们正训练一个2000亿参数的MoE模型(400亿活跃参数),初步结果已显示显著提升,进一步验证了AutoThink范式的可扩展性。
原文摘要 · Abstract (English)
We present Kwaipilot-AutoThink (KAT), an open-source 40B large language model developed to address the overthinking problem in reasoning-intensive tasks, where an automatic thinking training paradigm is proposed to dynamically switch between reasoning and non-reasoning modes based on task complexity. Specifically, first, we construct the dual-regime dataset based on a novel tagging pipeline and a multi-agent synthesis strategy, and then we apply Multi-Token Prediction (MTP)-enhanced knowledge distillation, enabling efficient and fine-grained reasoning transfer with minimal pretraining cost. Besides, we implement a cold-start initialization strategy that introduces mode-selection priors using majority-vote signals and intent-aware prompting. Finally, we propose Step-SRPO, a reinforcement learning algorithm that incorporates intermediate supervision into the GRPO framework, offering structured guidance over both reasoning-mode selection and response accuracy. Extensive experiments across multiple benchmarks demonstrate that KAT consistently matches or even outperforms current state-of-the-art models, including DeepSeek-R1-0528 and Qwen3-235B-A22B, across a wide range of reasoning-intensive tasks while reducing token usage. Notably, KAT outperforms all open-source models and even surpasses o3-mini on the leakage-controlled LiveCodeBench Pro. Beyond academic evaluation, KAT has been successfully deployed in Kwaipilot (i.e., Kuaishou's internal coding assistant), where it improves real-world development workflows with high accuracy, efficiency, and controllable reasoning behaviors. Moreover, we are actively training a 200B Mixture-of-Experts (MoE) model with 40B active parameters, and early results already show significant gains, further demonstrating the scalability of the AutoThink paradigm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。