超大模型HyperCLOVA X THINK专攻推理，支持双语高质量输出。
HyperCLOVA X THINK Technical Report
- 分三阶段训练，上下文窗口达128K，优化推理能力。
- 在韩语基准测试中表现媲美同类模型，且训练耗能更低。
- 适合需要强推理与双语一致性的韩国AI研发及开源应用。
我们介绍HyperCLOVA X THINK，这是首个专注于推理的HyperCLOVA X系列大语言模型，基于约6万亿高质量韩英语料预训练，并通过针对性合成韩语数据增强。采用计算-内存平衡的Peri-LN Transformer架构，以μP扩展规模，经三阶段课程训练逐步扩大上下文窗口至128K tokens；后通过监督微调与可验证奖励强化学习进行优化，支持详细推理链与简洁回答两种模式。在韩语聚焦基准如KMMLU、CSAT、KoBALT-700、HAERAE-1.0和KoBigBench上表现优异，同时保持出色的双语一致性与翻译质量。此外，视觉增强版本在KCSAT STEM基准上达到或超过GPT-4.1水平，且训练所需计算资源显著低于同规模模型。我们还提出一种剪枝与蒸馏技术，将用于后续开放源代码与商业友好型基础模型部署。整体能力使HyperCLOVA X THINK成为韩语AI创新的坚实基础，为全球研究社区提供宝贵资源。
原文摘要 · Abstract (English)
We introduce HyperCLOVA X THINK, the first reasoning-focused large language model in the HyperCLOVA X family, pre-trained on roughly $6$ trillion high-quality Korean, and English tokens, augmented with targeted synthetic Korean data. It was implemented as a compute-memory-balanced Peri-LN Transformer scaled with $μ$P, pre-trained through a three-stage curriculum that expands the context window to $128$K tokens, and post-trained via supervised fine-tuning with Reinforcement Learning from Verifiable Rewards supports both detailed rationale and concise-answer modes. It delivers competitive performance against similarly sized models on Korea-focused benchmarks such as KMMLU, CSAT, KoBALT-700, HAERAE-1.0, and KoBigBench, while preserving robust bilingual consistency and translation quality. In addition, a vision-augmented variant matches or exceeds GPT-4.1 on the KCSAT STEM benchmark, all of which are achieved with substantially lower training compute than existing models of similar sizes. We also present a pruning and distillation technique that will soon be applied to HyperCLOVA X THINK for an open-source and business-friendly foundation model. Altogether, these capabilities position HyperCLOVA X THINK as a robust foundation for Korean AI innovation and a valuable resource for the global research community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。