arXiv:2510.11407cs.CLcs.AI2025-10被引 4

让大模型学会识别自己知道与不知道,提升可信度。

KnowRL: Teaching Language Models to Know What They Know

  • 用模型自生成数据训练自我认知,无需外部标注。
  • 在两个模型上实现最高28%准确率提升,F1增12%。
  • 适合追求高可靠性、安全部署的AI应用开发者。

真正可靠的AI不仅需要海量知识,更需具备自我认知能力——知晓自身所知与未知。但现有研究显示,即使顶尖大模型在超过五分之一情况下也会误判自身能力,导致不可信输出。受自改进强化学习启发,我们提出简单而强大的框架KnowRL,增强模型对自身可行性边界的内省能力,从而实现更安全、负责任的行为。该框架包含两项核心:(i) 内省,模型自主判断任务是否可行;(ii) 基于共识的奖励机制,通过内部一致性强化自我认知稳定性。利用自生成数据,完全避免昂贵外部监督。在LLaMA-3.1-8B和Qwen-2.5-7B上的实验表明,仅需少量初始样本且无外部监督,KnowRL显著提升自我认知一致性,内在自洽性与外在基准测试均验证有效。经过数次迭代,准确率最高提升28%,F1提升12%,优于所有基线。该方法充分释放大模型自我提升知识意识的潜力,为可信赖、可问责的AI及关键场景安全部署铺路。因其简洁性与无需外部投入,我们建议将此可靠性增强流程应用于所有未来模型。

原文摘要 · Abstract (English)

Truly reliable AI requires more than simply scaling up knowledge; it demands the ability to know what it knows and when it does not. Yet recent research shows that even the best LLMs misjudge their own competence in more than one in five cases, making any response born of such internal uncertainty impossible to fully trust. Inspired by self-improvement reinforcement learning techniques that require minimal data, we present a simple but powerful framework KnowRL that strengthens a model's internal understanding of its own feasibility boundaries, enabling safer and more responsible behaviour. Our framework combines two components: (i) introspection, where the model generates and classifies tasks it judges feasible or infeasible, and (ii) consensus-based rewarding, where stability of self-knowledge assessment is reinforced through internal agreement. By using internally generated data, this design strengthens consistency in self-knowledge and entirely avoids costly external supervision. In experiments on LLaMA-3.1-8B and Qwen-2.5-7B, KnowRL steadily improved self-knowledge, validated by both intrinsic self-consistency and extrinsic benchmarking. With nothing more than a small seed set and no external supervision, our method drove gains as high as 28% in accuracy and 12% in F1, outperforming baselines in just a few iterations. Our framework essentially unlocks the untapped capacity of LLMs to self-improve their knowledge awareness, opening the door to reliable, more accountable AI and safer deployment in critical applications. Owing to its simplicity and independence from external effort, we encourage applying this reliability-enhancing process to all future models.

自我认知大模型可靠性自监督AI安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。