用对抗生成小数据集，让大模型推理更准更省
NanoFlux: Adversarial Dual-LLM Evaluation and Distillation For Multi-Domain Reasoning
- 两模型对抗生成针对性训练数据，带解释性多步问题
- 200条数据提升数学/科学/医疗推理，最高增16.6%
- 自动筛选新样本，适合想高效优化推理能力的研究者
我们提出NanoFlux，一种新型对抗框架，通过生成少于200个样本的针对性训练数据，显著提升大语言模型的跨领域推理能力。该框架在攻击者与防御者角色间交替迭代，由工具增强的裁判监督，合成包含解释注释的多步问题，聚焦特定推理能力。在4B参数模型上使用NanoFlux生成的数据微调，相比全基准微调，在数学推理（GSMHard）上提升+5.9%，科学推理（GenomeBench）+3.6%，医疗推理（MultiMedQA）+16.6%，同时计算开销降低3-14倍。消融实验揭示数据特征与性能间非单调关系，发现各领域最优的问题复杂度与推理质量点。NanoFlux通过嵌入式新颖性过滤、工具增强评估和多跳推理，实现训练数据自动生成，暗示未来模型优化可能依赖智能合成的小而精准数据集。
原文摘要 · Abstract (English)
We present NanoFlux, a novel adversarial framework for generating targeted training data to improve LLM reasoning, where adversarially-generated datasets containing fewer than 200 examples outperform conventional fine-tuning approaches. The framework employs a competitive dynamic between models alternating as Attacker and Defender, supervised by a tool-augmented Judge, synthesizing multi-step questions with explanatory annotations that target specific reasoning capabilities. Fine-tuning a 4B-parameter model on NanoFlux-generated data yields performance gains across diverse domains compared to full-benchmark fine-tuning: +5.9% on mathematical reasoning (GSMHard), +3.6% on scientific reasoning (GenomeBench), and +16.6% on medical reasoning (MultiMedQA), while reducing computational requirements by 3-14x. Ablation studies reveal a non-monotonic relationship between dataset characteristics and model performance, uncovering domain-specific optimal points for question complexity and reasoning quality. NanoFlux automates training data generation through embedding-based novelty filtering, tool-augmented evaluation, and multi-hop reasoning, suggesting that future model improvements may lie in the intelligent synthesis of small, precisely targeted training datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。