让模型从多位思考者中学习推理，提升小样本下的泛化能力
Learning to Think from Multiple Thinkers
- 设计主动学习算法，仅需少量每位思考者的思维链数据
- 所需思考者数量仅需约 log(1/ε)·log log(1/ε)，与精度要求弱相关
- 适用于推理任务中仅有结果标签而无完整推理过程的场景
我们研究从多位思考者获取的链式思维（CoT）监督信号进行学习，这些思考者虽都给出正确解法，但推理路径可能系统性不同。针对仅靠最终结果监督难以学习、但单个思考者提供CoT后可轻松学习的类别，我们证明：在被动数据收集设定下，若仅由两个或少数几位思考者提供CoT，则学习可能是计算上困难的（基于密码学假设）。相反，我们提出一种通用的高效主动学习算法，可在每名思考者仅需少量CoT数据的前提下，以独立于目标精度ε的样本量完成学习；所需思考者数量为O(log(1/ε)·log log(1/ε))，被动端结果数据需求为O(1/ε · poly log(1/ε))。
原文摘要 · Abstract (English)
We study learning with Chain-of-Thought (CoT) supervision from multiple thinkers, all of whom provide correct but possibly systematically different solutions, e.g., step-by-step solutions to math problems written by different thinkers, or step-by-step execution traces of different programs solving the same problem. We consider classes that are computationally easy to learn using CoT supervision from a single thinker, but hard to learn with only end-result supervision, i.e., without CoT (Joshi et al. 2025). We establish that, under cryptographic assumptions, learning can be hard from CoT supervision provided by two or a few different thinkers, in passive data-collection settings. On the other hand, we provide a generic computationally efficient active learning algorithm that learns with a small amount of CoT data per thinker that is completely independent of the target accuracy $\varepsilon$, a moderate number of thinkers that scales as $\log \frac{1}{\varepsilon}\log \log \frac{1}{\varepsilon}$, and sufficient passive end-result data that scales as $\frac{1}{\varepsilon}\cdot poly\log\frac{1}{\varepsilon}$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。