通过群体一致性强化学习，提升大模型推理的稳定性和准确性。
CC-LEARN: Cohort-based Consistency Learning
- 基于相似问题群组设计强化学习目标，引导模型统一推理模式。
- 在ARC-Challenge等挑战性基准上，准确率和推理稳定性显著优于基线。
- 适合关注大模型可靠推理、需要减少幻觉与不一致性的研究者。
大语言模型在诸多任务中表现优异，但在一致且稳健的推理方面仍存在不足。本文提出群体一致性学习（CC-Learn），一种基于强化学习的框架，通过共享程序抽象生成的相似问题群组进行训练，提升模型推理可靠性。为确保群体层面的一致性，定义了复合目标：包含群体准确率、用于有效问题分解的检索奖励，以及对无效或简单查找的拒绝惩罚，该目标可由强化学习直接优化，而监督微调无法实现。优化该奖励函数促使模型在所有同群问题中采用一致的推理路径。在多个高难度推理基准（包括ARC-Challenge和StrategyQA）上的实验表明，相较于预训练模型和监督微调基线，CC-Learn显著提升了准确率与推理稳定性。结果证明，群体级强化学习能有效增强大模型的推理一致性。
原文摘要 · Abstract (English)
Large language models excel at many tasks but still struggle with consistent, robust reasoning. We introduce Cohort-based Consistency Learning (CC-Learn), a reinforcement learning framework that improves the reliability of LLM reasoning by training on cohorts of similar questions derived from shared programmatic abstractions. To enforce cohort-level consistency, we define a composite objective combining cohort accuracy, a retrieval bonus for effective problem decomposition, and a rejection penalty for trivial or invalid lookups that reinforcement learning can directly optimize, unlike supervised fine-tuning. Optimizing this reward guides the model to adopt uniform reasoning patterns across all cohort members. Experiments on challenging reasoning benchmarks (including ARC-Challenge and StrategyQA) show that CC-Learn boosts both accuracy and reasoning stability over pretrained and SFT baselines. These results demonstrate that cohort-level RL effectively enhances reasoning consistency in LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。