教大模型认清自己能力边界，提升判断力和可靠性。
Capability Self-Assessment: Teaching LLMs to Know Their Limits
- 用强化学习训练模型自我评估能力，不牺牲原有性能。
- 相比监督微调，强化学习显著提升自我评估准确率。
- 可推广至新任务，适合需要可靠决策的场景。
识别自身局限并决定是否自行解决或委托是可靠智能系统的关键能力。然而我们发现，现代大语言模型普遍存在能力高估问题：在不同模型家族与规模下，它们会尝试无法解决的任务。我们将此能力称为能力自我评估（CSA），并将其建模为策略学习问题，旨在提升自我评估能力的同时保持原有能力。实验表明，强化学习能有效训练出良好的自我评估行为，显著优于监督微调，且不损害原模型性能；而监督微调则严重削弱模型本应具备的能力。此外，学习到的自我评估具有良好的分布外泛化能力，说明其是一种可迁移的模型特性。最后，该能力在推理时提升本地-云端任务决策效率，并可用于训练阶段的针对性数据筛选。
原文摘要 · Abstract (English)
The ability to recognize one's own limitations and decide whether to solve a problem or delegate is fundamental for reliable intelligent systems. Yet we show that modern large language models systematically lack this ability: across diverse model families and scales, they overestimate their competence and attempt queries they cannot solve. We refer to this ability as Capability Self-Assessment (CSA) and formulate it as a policy-learning problem, aiming to improve self-assessment while preserving the model's original capabilities. Our results show that reinforcement learning teaches CSA effectively, significantly outperforming supervised fine-tuning while preserving original capabilities. In contrast, supervised fine-tuning severely degrades the capabilities the model is meant to assess. Moreover, learned self-assessment behavior generalizes well out of distribution, suggesting that CSA is a transferable model trait. Finally, CSA is practically useful: it improves local-cloud decision making at inference time and provides a signal for targeted data selection during training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。