arXiv:2606.16811cs.AIcs.CL2026-06被引 1

用少量标注数据训练验证器,自动筛选高质量推理路径,显著降低标注成本。

Scaling LLM Reasoning from Minimal Labels: A Semi-Supervised Framework with a Lightweight Verifier

论文配图:Scaling LLM Reasoning from Minimal Labels: A Semi-Supervised Framework with a Lightweight Verifier
图 1 · 摘自论文原文
  • 用少量标签训练轻量验证器,判断大模型生成的推理过程是否正确。
  • 在数学和视觉问答任务上,仅需1/10~1/15的标注数据即达相近准确率。
  • 适合资源有限但需构建大规模推理能力的团队,尤其适合自动化系统训练。

在大语言模型发展过程中,伪中间推理生成方法进展显著,但通常依赖大量标注答案来评估推理质量。本文提出一种半监督框架,实现从极少量监督中扩展推理学习,将推理验证本身转化为数据生成机制。我们仅用少量标注样本训练一个轻量级推理正确性分类器,用于判断大模型生成的中间推理轨迹是否有效。此外,基于熵的置信度阈值剔除不可靠样本,剩余高置信度推理轨迹用于微调模型。在可验证数学问题(Orca-Math子集)和基于图像场景图的问题回答(GQA with Visual Programming)上的实验表明,本方法在仅使用10-15倍更少标注数据的情况下,仍能达到相近准确率。消融分析证实,分类器与熵过滤对可扩展且抗噪的伪标签生成至关重要。通过以轻量级推理验证替代昂贵的答案级监督,本方法为构建大规模推理资源提供了可行路径,并为未来从极少人类输入中自主学习的推理系统铺平道路。

原文摘要 · Abstract (English)

For the development of Large language models (LLMs), recent approaches to generating pseudo intermediate reasoning have shown remarkable progress. But they typically rely on large numbers of correctly annotated answers to assess reasoning quality. This paper presents a semi-supervised framework that scales reasoning learning from minimal supervision, turning reasoning verification itself into a data creation mechanism. We train a lightweight reasoning-correctness classifier on only a few labeled samples, which judges whether intermediate reasoning traces generated by an LLM are valid. Furthermore, an entropy-based confidence threshold filters out unreliable samples, and the remaining high-confidence reasoning traces are used to fine-tune the model. Experiments on Verifiable Math Problems (Orca-Math subset) and Question Answering on Image Scene Graphs (GQA) with Visual Programming show that our method achieves accuracy comparable to using 10-15x more labeled data. Ablation analyses confirm that both the classifier and entropy filtering are essential for scalable and noise-resistant pseudo-labeling. By replacing expensive answer-level supervision with lightweight reasoning verification, our method provides a practical path toward constructing large-scale reasoning resources and paves the way for future autonomous reasoning systems that learn from minimal human input.

大模型推理半监督学习轻量验证器数据高效

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。