用自生成问题训练模型,让大模型学会系统思考。
MindGYM: What Matters in Question Synthesis for Thinking-Centric Fine-Tuning?
- 通过注入高阶推理目标,引导模型自主生成有逻辑的问题。
- 合成数据质量提升16.7%,方差降低67.91%,更稳定可靠。
- 仅用400条数据提升16%推理性能,适合轻量级高效训练。
大模型在获取可迁移、结构化的思维能力方面面临挑战,尤其当使用固定模板或众包标注的指令数据集时。不同于以往方法,本文提出以思维为中心的数据合成范式,使模型通过自我生成的认知引导数据实现进化。我们设计了MindGYM框架,包含:(1) 认知思维过程注入,将高层次推理目标融入模型生成行为;(2) 种子单跳问题生成,从多样化语义类型中生成原子性问题,促进更广泛思维;(3) 挑战性多跳问答生成,基于问答种子构建更复杂的多跳问题以深化推理。分析显示,该方法生成的合成数据平均质量提升16.7%,质量方差降低67.91%,证明高质量且自洽的数据对思维导向微调至关重要。MindGYM在六个推理基准上均表现优异,仅用400个样本即在MathVision上取得最高16%的提升,并在不同模型规模与架构间具泛化性。该工作验证了自挑战机制在减少人工干预和资源消耗的同时,有效提升大模型能力。代码与数据已开源,推动基于内部推理能力驱动的自演化基础模型研究。
原文摘要 · Abstract (English)
Large foundation models face challenges in acquiring transferable, structured thinking abilities, especially when supervised with rigid templates or crowd-annotated instruction datasets. Unlike prior approaches, we focus on a thinking-centric data synthesis paradigm that enables models to evolve through self-generated, cognitively guided data. We propose MindGYM, a structured and scalable framework for question synthesis, composed of: (1) Cognitive Thinking Process Injection, which infuses high-level reasoning objectives to shape the model's synthesis behavior; (2) Seed Single-Hop Question Synthesis, generating atomic questions from diverse semantic types to encourage broader thinking; and (3) Challenging Multi-Hop QA Synthesis, composing more complex multi-hop questions based on QA seeds for deeper reasoning. Detailed analysis shows that synthetic data generated by our method achieves 16.7% higher average quality and 67.91% lower quality variance compared to baseline sources, highlighting that both high-quality and self-contained data are essential for effective, thinking-oriented fine-tuning. MindGYM improves performance on six reasoning benchmarks, achieving gains of up to 16% on MathVision using only 400 data samples, and generalizable improvements across different model sizes and architectures. MindGYM underscores the viability of self-challenging mechanisms in refining large model capabilities while minimizing human intervention and resource demands. Code and data are released to promote data-centric research into self-evolving foundation models driven by their internal reasoning capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。